CN109446160A - A kind of file reading, system, device and computer readable storage medium - Google Patents
A kind of file reading, system, device and computer readable storage medium Download PDFInfo
- Publication number
- CN109446160A CN109446160A CN201811312842.5A CN201811312842A CN109446160A CN 109446160 A CN109446160 A CN 109446160A CN 201811312842 A CN201811312842 A CN 201811312842A CN 109446160 A CN109446160 A CN 109446160A
- Authority
- CN
- China
- Prior art keywords
- file
- metadata
- hash
- file destination
- memory address
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Storage Device Security (AREA)
Abstract
This application discloses a kind of file readings, the corresponding table of the Hash of corresponding relationship between the cryptographic Hash of same file and the direct memory address of the metadata of file is stored with by establishing, it is able in the case where not destroying the storage mode of directory tree of existing file system use, the direct memory address that its metadata only quickly, is directly determined by the absolute path of file destination only needs that the purpose for reading file destination can be completed according to the metadata read under the direct memory address later.No matter the store path depth of file destination has mostly deep, and few read operation all need to be only executed to hard disk can be completed identical purpose, is minimized read operation because executing excessive number influence caused by system performance.The application further simultaneously discloses a kind of file and reads system, device and computer readable storage medium, has above-mentioned beneficial effect.
Description
Technical field
This application involves data access technology field, in particular to a kind of file reading, system, device and computer
Readable storage medium storing program for executing.
Background technique
Data are stored in a manner of file in file system, and file is usually by real data and metadata group
Be exactly the data content that user really stores at, real data, metadata is to describe the data of real data, including how
The data such as the real data for finding this document, the variation of real data for how recording this document.
Existing file system manages file in the form of directory tree.Each catalogue is also a special file, each mesh
The file for including several subdirectories in record and being stored under current directory.When searching some file, need from top mesh
Record starts, and first looks for the metadata of top-level director, and the real data of top-level director is found according to the metadata of top-level director,
Confirm further according to the real data of top-level director and whether there is subdirectory or other files under it, if there is continuing to search son
The metadata of catalogue and file, then further according to the metadata lookup of subdirectory or file subdirectory/file real data, with
This analogizes, target directory or catalogue file until inquiry malfunctions or finds.
It should be noted that according to the directory tree storage mode that metadata under existing file system and real data use,
Every level-one directory search is required to execute read operation twice (metadata once adds real data primary) to disk, if target
The store path depth of file or target directory is deeper, to disk execute read operation number be even as high as it is tens of or up to a hundred, also
Seriously affect the performance of system.
Therefore, how to overcome the technological deficiency for reading file processes storage under existing file system, a kind of pair of magnetic is provided
Disk executes the less file reading mechanism of read operation number, is those skilled in the art's urgent problem to be solved.
Summary of the invention
The purpose of the application is to provide a kind of file reading, and the cryptographic Hash and text of same file are stored with by establishing
The Hash of corresponding relationship corresponds to table between the direct memory address of the metadata of part, is able to do not destroying the use of existing file system
Under the storage mode of directory tree, the direct storage of its metadata only quickly, is directly determined by the absolute path of file destination
Address only needs that the purpose for reading file destination can be completed according to the metadata read under the direct memory address later.
No matter the store path depth of file destination has mostly deeply, all only few read operation of hard disk execution need to can be completed identical
Purpose is minimized read operation because executing excessive number influence caused by system performance.
The another object of the application is the provision of a kind of file and reads system, device and computer readable storage medium.
To achieve the above object, the application provides a kind of file reading, and this document read method includes:
Obtain the absolute path of file destination；
The cryptographic Hash of the absolute path is calculated by hash algorithm；
The direct storage ground that table determines the metadata of the file destination is corresponded to according to the cryptographic Hash and preset Hash
Location；Wherein, the Hash corresponds to the direct memory address that table record has the cryptographic Hash of each file and the metadata of each file
Between corresponding relationship；
The metadata is read under the direct memory address, and the file destination is read according to the metadata
Real data.
Optionally, this document read method further include:
Choose the absolute path that store path depth is more than the file of predetermined depth；
The cryptographic Hash of each absolute path is calculated separately by the hash algorithm；
Establish the corresponding relationship between the cryptographic Hash of same file and the direct memory address of the metadata of file；
The Hash, which is established, according to each corresponding relationship corresponds to table.
Optionally, before the absolute path for obtaining file destination, further includes:
Inquiry whether there is the metadata of the file destination in the buffer；Wherein, it is previously stored in the caching
The metadata of the file accessed in prefixed time interval；
If being stored with the metadata of the file destination in the caching, according to the target text being stored in the caching
The metadata of part reads the real data of the file destination；
If the not stored metadata for having the file destination in the caching, the absolute path for obtaining file destination is executed
The step of.
Optionally, this document read method further include:
The all information that the Hash corresponds to table is dispersed to multiple memory nodes；
It is corresponding, according to the cryptographic Hash and preset Hash correspond to table determine the file destination metadata it is direct
Storage address, comprising:
It is corresponded to according to the cryptographic Hash in the Hash and determines target storage node in table；
The direct memory address of the metadata of the file destination is checked in the target storage node.
To achieve the above object, present invention also provides a kind of files to read system, and this document reads system and includes:
Absolute path acquiring unit, for obtaining the absolute path of file destination；
Cryptographic Hash computing unit, for the cryptographic Hash of the absolute path to be calculated by hash algorithm；
Direct memory address determination unit determines the target for corresponding to table according to the cryptographic Hash and preset Hash
The direct memory address of the metadata of file；Wherein, the Hash, which corresponds to table record, the cryptographic Hash of each file and each text
Corresponding relationship between the direct memory address of the metadata of part；
File destination reading unit, for reading the metadata under the direct memory address, and according to the member
Real data of the reading data to the file destination.
Optionally, this document reads system further include:
File selection unit, for choosing absolute path of the store path depth more than the file of predetermined depth；
Cryptographic Hash pre-calculation unit, for calculating separately the Hash of each absolute path by the hash algorithm
Value；
Correspondence relationship establishing unit, for establishing the direct memory address of the cryptographic Hash of same file and the metadata of file
Between corresponding relationship；
Hash corresponds to table and establishes unit, corresponds to table for establishing the Hash according to each corresponding relationship.
Optionally, this document reads system further include:
Caching query unit, for before the absolute path for obtaining file destination, inquiry to whether there is institute in the buffer
State the metadata of file destination；Wherein, the member of the file accessed within a preset time interval is previously stored in the caching
Data；
Cache metadata reading unit, for when being stored with the metadata of the file destination in the caching, according to
The metadata for the file destination being stored in the caching reads the real data of the file destination；
Absolute path obtains execution unit, for when the not stored metadata for having the file destination in the caching,
The step of executing the absolute path for obtaining file destination.
Optionally, this document reads system further include:
Information dispersion unit, all information for the Hash to be corresponded to table are dispersed to multiple memory nodes；
Corresponding, the direct memory address determination unit includes:
Target storage node determines subelement, determines that target is deposited in table for corresponding to according to the cryptographic Hash in the Hash
Store up node；
Direct memory address inquires subelement, for checking in first number of the file destination in the target storage node
According to direct memory address.
To achieve the above object, present invention also provides a kind of document reading apparatus, this document reading device includes:
Memory, for storing computer program；
Processor realizes the step of the file reading as described in above content when for executing the computer program
Suddenly.
To achieve the above object, described computer-readable to deposit present invention also provides a kind of computer readable storage medium
It is stored with computer program on storage media, the text as described in above content is realized when the computer program is executed by processor
The step of part read method.
Obviously, a kind of file reading provided herein, by establish be stored with the cryptographic Hash of same file with
The Hash of corresponding relationship corresponds to table between the direct memory address of the metadata of file, is able to do not destroying the use of existing file system
Directory tree storage mode under, directly depositing for its metadata only quickly, is directly determined by the absolute path of file destination
Address is stored up, only needs that the mesh for reading file destination can be completed according to the metadata read under the direct memory address later
's.No matter the store path depth of file destination has mostly deeply, and few read operation all need to be only executed to hard disk can be completed phase
With purpose, be minimized read operation because executing excessive number influence caused by system performance.
The application additionally provides a kind of file simultaneously and reads system, device and computer readable storage medium, has above-mentioned
Beneficial effect, details are not described herein.
Detailed description of the invention
In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of application for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of flow chart of file reading provided by the embodiments of the present application；
Fig. 2 is a kind of process established Hash and correspond to the method for table in file reading provided by the embodiments of the present application
Figure；
Fig. 3 is that a kind of caching that utilization is stored with part metadata is next in file reading provided by the embodiments of the present application
Reduce the flow chart that read operation executes the method for number；
Fig. 4 is a kind of structural schematic diagram of the corresponding relationship storage method based on linked list type provided by the embodiments of the present application；
Fig. 5 is the structural schematic diagram of minimum unit in chained list shown in Fig. 4；
Fig. 6 is the structural block diagram that a kind of file provided by the embodiments of the present application reads system.
Specific embodiment
The core of the application is to provide a kind of file reading, system, device and computer readable storage medium, passes through
Foundation is stored with the corresponding table of the Hash of corresponding relationship between the cryptographic Hash of same file and the direct memory address of the metadata of file,
It is able in the case where not destroying the storage mode of directory tree of existing file system use, it is only fast by the absolute path of file destination
Direct memory address that is fast, directly determining its metadata is only needed later according to the member read under the direct memory address
The purpose for reading file destination can be completed in data.No matter the store path depth of file destination has mostly deeply, all only need to be to hard disk
Executing few read operation can be completed identical purpose, be minimized the read operation because executing excessive number to system performance
Caused by influence.
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application
In attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is
Some embodiments of the present application, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art
All other embodiment obtained without making creative work, shall fall in the protection scope of this application.
Embodiment one
Referring to Figure 1, Fig. 1 is a kind of flow chart of file reading provided by the embodiments of the present application comprising following
Step:
S101: the absolute path of file destination is obtained；
The absolute path of file destination be catalogue where referring to directly to be accessed the file destination by this paths or
The path of position.With an image file being stored under catalogue entitled 003 (it is assumed that the entitled picture.jpg of its file)
For, the upper directory of catalogue 003 is catalogue 02, and the upper directory of catalogue 02 is root d :/, therefore the image file is exhausted
It is d :/02/003/picture.jpg to path.
The absolute path of file destination can obtain easily under the existing file system based on directory tree, such as directly receive
It is being converted from the background after the absolute path of the file destination of user's input, the relative path of the file destination of reception user's input
It extracts and obtains, etc. from file access request when obtaining, and being inquired using traditional approach, herein no longer sieve one by one
Column.
S102: the cryptographic Hash of absolute path is calculated by hash algorithm；
On the basis of S101, this step is intended to be calculated using hash algorithm and the unique corresponding Kazakhstan of the absolute path
Uncommon value.
Hash is got by English Hash transliteration, also referred to as " is hashed ", which can be the input of random length (again
It is called preliminary mapping pre-image) output of regular length is transformed by hashing algorithm, which is exactly that hashed value (is also referred to as breathed out
Uncommon value).Briefly, hash algorithm is a kind of letter of the eap-message digest of the message compression by random length to a certain regular length
Number.
The common hash algorithm used specifically includes: MD4, MD5 and SHA-1, wherein MD is English Message Digest
Abbreviation, Chinese is eap-message digest, and MD5 and MD4 are the version of two different times respectively, MD5 is more complicated relative to MD4,
It is safer；SHA-1 is to all length less than 264Input information, can generate length be 160bit hashed value, have more
Good anti-exhaustive.MD5 value or characteristic value are also referred to as based on the cryptographic Hash that MD5 algorithm calculates.Using in this step
When, most suitable hash algorithm is selected in combination with the information of concrete application scene, herein and is not specifically limited.
S103: the direct memory address that table determines the metadata of file destination is corresponded to according to cryptographic Hash and preset Hash；
On the basis of S102, this step is intended to correspond to the cryptographic Hash pair that table is checked in and is calculated by preset Hash
" value " answered, wherein the Hash correspond in table " value " corresponding with each cryptographic Hash be specially the file destination metadata
Direct memory address, the i.e. Hash, which correspond to record in table, has the cryptographic Hash of each file to deposit with the direct of the metadata of each file
Store up the corresponding relationship between address.
It should be noted that direct memory address is the storage address under catalogue where the metadata, mesh where not including
Other interference informations such as the upper directory of record, therefore can directly just be obtained under the direct memory address by a read operation should
The metadata of file destination, without as needing to go successively to find next stage catalogue by the catalogue of level-one grade in the prior art
Metadata can effectively reduce the read operation number of hard disk execution.
S104: reading metadata under from direct memory address, and the real data of file destination is read according to metadata.
On the basis of S103, this step, which is intended to correspond to from Hash, reads storage under the direct memory address checked in table
The metadata of file destination, and the real data for reading according to the metadata file destination so far completes file destination
Read operation.It therefore deduces that, the file reading provided using the application, for each Hash by its absolute path
Value and the direct memory address of metadata are stored in the file in the corresponding table of Hash, can it is unrelated its under file system now
In the case where store path depth, by most successfully reading the file destination to the read operation of hard disk twice.
Based on the above-mentioned technical proposal, file reading provided by the embodiments of the present application is stored with phase identical text by establishing
The corresponding table of the Hash of corresponding relationship between the direct memory address of the metadata of the cryptographic Hash and file of part, be able to do not destroy it is existing
Under the storage mode for the directory tree that file system uses, only by the absolute path of file destination quickly, directly determine its yuan
The direct memory address of data only needs that reading mesh can be completed according to the metadata read under the direct memory address later
Mark the purpose of file.No matter the store path depth of file destination has mostly deeply, and few read operation all need to be only executed to hard disk
Identical purpose can be completed, be minimized read operation because executing excessive number influence caused by system performance.
Embodiment two
Fig. 2 is referred to, Fig. 2 is a kind of side for establishing Hash and corresponding to table in file reading provided by the embodiments of the present application
The flow chart of method, the application are intended to provide a kind of method established Hash and correspond to table, so as to can normally make in file system
With, completing to include the following steps: the read operation of file destination in the process by way of embodiment one
S201: the absolute path that store path depth is more than the file of predetermined depth is chosen；
For the influence reduced to system performance as maximized as possible, the present embodiment is more than by store path depth is chosen
The absolute path of the file of predetermined depth, rather than the absolute path of the file of all store path depth, because in root
Under file destination read operation twice need to be only executed according to the prior art, i.e., according to the prior art read be located at root
Under file destination influence to be consistent with according to technical solution provided by the present application caused by system performance, difference is that
If the deeper file of store path depth is read out according to the prior art, read operation execute number will with pathdepth at
Direct ratio.
Therefore, which can be also configured as the second depth (subdirectory or file i.e. under root) and be deeper than second
Other depth of depth, because while the execution number of read operation can effectively reduce by technical solution provided by the present application, but
Building Hash correspond to table, table look-up, calculates cryptographic Hash be also required to occupy part system performance, it is therefore necessary to weigh the two to system
It is influenced caused by performance, and selects most suitable scheme.
S202: the cryptographic Hash of each absolute path is calculated separately by hash algorithm；
S203: the corresponding relationship between the cryptographic Hash of same file and the direct memory address of the metadata of file is established；
S204: Hash is established according to each corresponding relationship and corresponds to table.
Embodiment three
Fig. 3 is referred to, Fig. 3 is stored with part member number for utilization a kind of in file reading provided by the embodiments of the present application
According to caching come reduce read operation execute number method flow chart, based on any of the above embodiments, the present embodiment
Another way is intended to provide to accelerate the speed of reading file destination as far as possible, i.e., by the member of storage section file
The cachings of data is realized, is included the following steps:
S301: the access request of file destination is received；
S302: inquiry whether there is the metadata of file destination in the buffer；
S303: the real data of file destination is read according to the metadata for the file destination being stored in caching；
This step is established on the basis of the query result of S302 is to be stored with the metadata of the file destination in caching, because
This can directly read the real data of file destination according to the metadata for the file destination being stored in caching.
S304: the absolute path of file destination is obtained.
This step is established on the basis of the query result of S302 is the not stored metadata for having the file destination in caching,
Therefore it just needs to start to execute according to the step of one S101 of embodiment.Specifically, when including in the access request that S301 is received
When having the absolute path of the file destination, it can directly extract and obtain from the access request；When the access that S301 is received is asked
When in asking not including the absolute path of the file destination, can directly to user request, or receive user input again it is exhausted
To path.
It should be noted that a part of caching as CPU (central processing unit), data access speed is very fast, but its
Memory capacity is smaller, further, can also be pre-stored in the metadata of partial document in the bigger memory of memory capacity,
Because while the data access speed relative cache of memory is slower, but it is obvious excellent relative to directly reading or having from hard disk
Gesture.
Example IV
In order to deepen the understanding to the present application point, the present embodiment will provide a kind of specific in conjunction with concrete application scene
Implementation under scene:
Hash is established on disk first and corresponds to table, the file that store path depth is greater than the set value is chosen, with this document
Absolute path carry out Hash operation, thus by the address of disk block where the cryptographic Hash being calculated and the metadata of this document
It stores the Hash to correspond in table, wherein calculated cryptographic Hash is uniquely corresponding with absolute path.The present embodiment uses multiple disks
One complete Hash of maintenance corresponds to table jointly, i.e., each disk will correspond to a Hash section of table as the complete Hash of composition
Point is stored with the address of each metadata saved on the Hash node on each Hash node, therefore according to target text
When the absolute path of part calculates cryptographic Hash, first have to judge information storage relevant to the cryptographic Hash according to calculated cryptographic Hash
In which Hash node, and first number of the file destination is read after confirming target Hash node from target Hash node
According to.
It calculates by being usually the disk number (Hash that the cryptographic Hash being actually calculated corresponds to the complete Hash of composition to table
Number of nodes) remainder is carried out, to obtain target Hash node according to remainder, multiple corresponding relationships in same Hash node can be adopted
It is stored with chained list as shown in Figure 4, wherein each chained list node stores the cryptographic Hash and its member number of file absolute path
According to storage address, as shown in Figure 5.
The present invention carries out the absolute path for the file that file path depth is greater than the set value when searching file destination
Hash is calculated, and obtains corresponding hash value；Hash value is to node maximum value remainder, to find corresponding in hash table
Hash node, and the corresponding chained list node of hash value of file is found in the chained list of corresponding hash node in turn.Further according to
Chained list node finds the block of storage metadata
Because situation is complicated, it can not enumerate and be illustrated, those skilled in the art should be able to recognize according to the application
The basic skills principle combination actual conditions of offer may exist many examples, in the case where not paying enough creative works,
It should within the scope of protection of this application.
Embodiment five
Fig. 6 is referred to below, and Fig. 6 is the structural block diagram that a kind of file provided by the embodiments of the present application reads system, this article
Part reads system
Absolute path acquiring unit 100, for obtaining the absolute path of file destination；
Cryptographic Hash computing unit 200, for the cryptographic Hash of absolute path to be calculated by hash algorithm；
Direct memory address determination unit 300 determines file destination for corresponding to table according to cryptographic Hash and preset Hash
Metadata direct memory address；Wherein, Hash, which corresponds to table record, the cryptographic Hash of each file and first number of each file
According to direct memory address between corresponding relationship；
File destination reading unit 400 is read for reading metadata under direct memory address, and according to metadata
The real data of file destination.
Further, this document, which reads system, to include:
File selection unit, for choosing absolute path of the store path depth more than the file of predetermined depth；
Cryptographic Hash pre-calculation unit, for calculating separately the cryptographic Hash of each absolute path by hash algorithm；
Correspondence relationship establishing unit, for establishing the direct memory address of the cryptographic Hash of same file and the metadata of file
Between corresponding relationship；
Hash corresponds to table and establishes unit, corresponds to table for establishing Hash according to each corresponding relationship.
Further, this document, which reads system, to include:
Caching query unit, for before the absolute path for obtaining file destination, inquiry to whether there is mesh in the buffer
Mark the metadata of file；Wherein, the metadata of the file accessed within a preset time interval is previously stored in caching；
Cache metadata reading unit, it is slow according to being stored in for when being stored with the metadata of file destination in caching
The metadata of file destination in depositing reads the real data of file destination；
Absolute path obtains execution unit, for executing acquisition when the not stored metadata for having file destination in caching
The step of absolute path of file destination.
Further, this document, which reads system, to include:
Information dispersion unit, all information for Hash to be corresponded to table are dispersed to multiple memory nodes；
Corresponding, direct memory address determination unit 300 may include:
Target storage node determines subelement, and target storage node is determined in table for corresponding to according to cryptographic Hash in Hash；
Direct memory address inquire subelement, for checked in target storage node file destination metadata it is direct
Storage address.
Based on the above embodiment, present invention also provides a kind of document reading apparatus, this document reading device may include
Memory and processor, wherein have computer program in the memory, which calls the computer journey in the memory
When sequence, step provided by above-described embodiment may be implemented.Certainly, this document reading device can also include various necessary nets
Network interface, power supply and other components etc..
Present invention also provides a kind of computer readable storage mediums, have computer program thereon, the computer program
Step provided by above-described embodiment may be implemented when being performed terminal or processor execution.The storage medium may include: U
Disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access
Memory, RAM), the various media that can store program code such as magnetic or disk.
Specific examples are used herein to illustrate the principle and implementation manner of the present application, and between each embodiment
For progressive relationship, each embodiment focuses on the differences from other embodiments, identical between each embodiment
Similar portion may refer to each other.For the device disclosed in the embodiment, reference can be made to corresponding method part illustration.The above reality
The explanation for applying example is merely used to help understand the present processes and its core concept.For the ordinary skill people of the art
Member for, under the premise of not departing from the application principle, can also to the application, some improvement and modification can also be carried out, these improve and
Modification is also fallen into the protection scope of the claim of this application.
It should also be noted that, in the present specification, relational terms such as first and second and the like be used merely to by
One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning
Covering non-exclusive inclusion, so that the process, method, article or equipment for including a series of elements not only includes that
A little elements, but also other elements including being not explicitly listed, or further include for this process, method, article or
The intrinsic element of equipment.In the absence of more restrictions, the element limited by sentence "including a ...", is not arranged
Except there is also other identical elements in the process, method, article or equipment for including element.
Claims (10)
1. a kind of file reading, which is characterized in that the file reading includes:
Obtain the absolute path of file destination；
The cryptographic Hash of the absolute path is calculated by hash algorithm；
The direct memory address that table determines the metadata of the file destination is corresponded to according to the cryptographic Hash and preset Hash；Its
In, the Hash corresponds to table and records pair having between the cryptographic Hash of each file and the direct memory address of the metadata of each file
It should be related to；
The metadata is read under the direct memory address, and the reality of the file destination is read according to the metadata
Border data.
2. file reading according to claim 1, which is characterized in that further include:
Choose the absolute path that store path depth is more than the file of predetermined depth；
The cryptographic Hash of each absolute path is calculated separately by the hash algorithm；
Establish the corresponding relationship between the cryptographic Hash of same file and the direct memory address of the metadata of file；
The Hash, which is established, according to each corresponding relationship corresponds to table.
3. file reading according to claim 1, which is characterized in that obtain file destination absolute path it
Before, further includes:
Inquiry whether there is the metadata of the file destination in the buffer；Wherein, it is previously stored in the caching default
The metadata of the file accessed in time interval；
If being stored with the metadata of the file destination in the caching, according to the file destination being stored in the caching
Metadata reads the real data of the file destination；
If the not stored metadata for having the file destination in the caching, the step for obtaining the absolute path of file destination is executed
Suddenly.
4. file reading according to any one of claims 1 to 3, which is characterized in that further include:
The all information that the Hash corresponds to table is dispersed to multiple memory nodes；
It is corresponding, the direct storage that table determines the metadata of the file destination is corresponded to according to the cryptographic Hash and preset Hash
Address, comprising:
It is corresponded to according to the cryptographic Hash in the Hash and determines target storage node in table；
The direct memory address of the metadata of the file destination is checked in the target storage node.
5. a kind of file reads system characterized by comprising
Absolute path acquiring unit, for obtaining the absolute path of file destination；
Cryptographic Hash computing unit, for the cryptographic Hash of the absolute path to be calculated by hash algorithm；
Direct memory address determination unit determines the file destination for corresponding to table according to the cryptographic Hash and preset Hash
Metadata direct memory address；Wherein, the Hash, which corresponds to table record, the cryptographic Hash of each file and each file
Corresponding relationship between the direct memory address of metadata；
File destination reading unit, for reading the metadata under the direct memory address, and according to the metadata
Read the real data of the file destination.
6. file according to claim 5 reads system, which is characterized in that further include:
File selection unit, for choosing absolute path of the store path depth more than the file of predetermined depth；
Cryptographic Hash pre-calculation unit, for calculating separately the cryptographic Hash of each absolute path by the hash algorithm；
Correspondence relationship establishing unit, for establishing between the cryptographic Hash of same file and the direct memory address of the metadata of file
Corresponding relationship；
Hash corresponds to table and establishes unit, corresponds to table for establishing the Hash according to each corresponding relationship.
7. file according to claim 5 reads system, which is characterized in that further include:
Caching query unit, for before the absolute path for obtaining file destination, inquiry to whether there is the mesh in the buffer
Mark the metadata of file；Wherein, the metadata of the file accessed within a preset time interval is previously stored in the caching；
Cache metadata reading unit, for when being stored with the metadata of the file destination in the caching, according to storage
The metadata of file destination in the caching reads the real data of the file destination；
Absolute path obtains execution unit, for executing when the not stored metadata for having the file destination in the caching
The step of obtaining the absolute path of file destination.
8. reading system according to the described in any item files of claim 5 to 7, which is characterized in that further include:
Information dispersion unit, all information for the Hash to be corresponded to table are dispersed to multiple memory nodes；
Corresponding, the direct memory address determination unit includes:
Target storage node determines subelement, and target storage section is determined in table for corresponding to according to the cryptographic Hash in the Hash
Point；
Direct memory address inquires subelement, for checking in the target storage node metadata of the file destination
Direct memory address.
9. a kind of document reading apparatus characterized by comprising
Memory, for storing computer program；
Processor realizes such as Claims 1-4 described in any item file readings when for executing the computer program
The step of.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program is realized when the computer program is executed by processor such as the described in any item file readings of Claims 1-4
Step.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811312842.5A CN109446160A (en) | 2018-11-06 | 2018-11-06 | A kind of file reading, system, device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811312842.5A CN109446160A (en) | 2018-11-06 | 2018-11-06 | A kind of file reading, system, device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109446160A true CN109446160A (en) | 2019-03-08 |
Family
ID=65551329
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811312842.5A Pending CN109446160A (en) | 2018-11-06 | 2018-11-06 | A kind of file reading, system, device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109446160A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110489701A (en) * | 2019-08-19 | 2019-11-22 | 安徽三实信息技术服务有限公司 | Extract the method, apparatus and CMS recognition methods of CMS identification feature |
| CN110647507A (en) * | 2019-09-12 | 2020-01-03 | 苏州浪潮智能科技有限公司 | File system writing state determining method and device, electronic equipment and medium |
| CN110795039A (en) * | 2019-10-25 | 2020-02-14 | 北京浪潮数据技术有限公司 | Data management method and device based on object storage service and related components |
| CN111769933A (en) * | 2020-06-29 | 2020-10-13 | 北京天融信网络安全技术有限公司 | Method and device for monitoring file change, electronic equipment and storage medium |
| CN111858075A (en) * | 2020-06-30 | 2020-10-30 | 广东浪潮大数据研究有限公司 | Deadlock detection method, system, equipment and readable storage medium |
| CN112256196A (en) * | 2020-10-20 | 2021-01-22 | 北京字跳网络技术有限公司 | File reading method and device, electronic equipment and computer readable medium |
| CN113127415A (en) * | 2019-12-31 | 2021-07-16 | 浙江宇视科技有限公司 | Method, device, medium and electronic equipment for processing real-time stream file |
| CN113420024A (en) * | 2021-06-02 | 2021-09-21 | 浪潮软件股份有限公司 | Business form data sub-table storage method |
| CN113625968A (en) * | 2021-08-12 | 2021-11-09 | 网易（杭州）网络有限公司 | File authority management method and device, computer equipment and storage medium |
| CN114442937A (en) * | 2021-12-31 | 2022-05-06 | 北京云宽志业网络技术有限公司 | File caching method and device, computer equipment and storage medium |
| CN114528258A (en) * | 2022-02-18 | 2022-05-24 | 北京百度网讯科技有限公司 | Asynchronous file processing method, device, server, medium, product and system |
| WO2022199400A1 (en) * | 2021-03-23 | 2022-09-29 | 中兴通讯股份有限公司 | Method and apparatus for retrieving persistent memory file system metadata, and storage structure |
| CN117493276A (en) * | 2024-01-03 | 2024-02-02 | 柏科数据技术（深圳）股份有限公司 | Reading method and device for Ceph file, server and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006058965A (en) * | 2004-08-17 | 2006-03-02 | Fujitsu Ltd | File management program, file management method, and file management device |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN101741731A (en) * | 2009-12-03 | 2010-06-16 | 中兴通讯股份有限公司 | Content metadata storing, inquiring method and managing system in content delivery network (CDN) |
| CN103229173A (en) * | 2012-12-26 | 2013-07-31 | 华为技术有限公司 | Metadata management method and system |
| CN108268208A (en) * | 2016-12-30 | 2018-07-10 | 清华大学 | A kind of distributed memory file system based on RDMA |
-
2018
- 2018-11-06 CN CN201811312842.5A patent/CN109446160A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006058965A (en) * | 2004-08-17 | 2006-03-02 | Fujitsu Ltd | File management program, file management method, and file management device |
| CN101692239A (en) * | 2009-10-19 | 2010-04-07 | 浙江大学 | Method for distributing metadata of distributed type file system |
| CN101741731A (en) * | 2009-12-03 | 2010-06-16 | 中兴通讯股份有限公司 | Content metadata storing, inquiring method and managing system in content delivery network (CDN) |
| CN103229173A (en) * | 2012-12-26 | 2013-07-31 | 华为技术有限公司 | Metadata management method and system |
| CN108268208A (en) * | 2016-12-30 | 2018-07-10 | 清华大学 | A kind of distributed memory file system based on RDMA |
Non-Patent Citations (3)
| Title |
|---|
| PETER F. CORBETT AND DROR G. FEITELSON: "The Vesta Parallel File System", 《ACM TRANSACTIONS ON COMPUTER SYSTEMS》 * |
| 方圆等: "基于对象存储的新型元数据管理策略", 《计算机工程》 * |
| 贺元香 等: "除留余数法建立哈希表的方法改进", 《甘肃科技》 * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110489701A (en) * | 2019-08-19 | 2019-11-22 | 安徽三实信息技术服务有限公司 | Extract the method, apparatus and CMS recognition methods of CMS identification feature |
| CN110647507A (en) * | 2019-09-12 | 2020-01-03 | 苏州浪潮智能科技有限公司 | File system writing state determining method and device, electronic equipment and medium |
| CN110795039A (en) * | 2019-10-25 | 2020-02-14 | 北京浪潮数据技术有限公司 | Data management method and device based on object storage service and related components |
| CN113127415A (en) * | 2019-12-31 | 2021-07-16 | 浙江宇视科技有限公司 | Method, device, medium and electronic equipment for processing real-time stream file |
| CN113127415B (en) * | 2019-12-31 | 2024-02-27 | 浙江宇视科技有限公司 | Real-time stream file processing method, device, medium and electronic equipment |
| CN111769933A (en) * | 2020-06-29 | 2020-10-13 | 北京天融信网络安全技术有限公司 | Method and device for monitoring file change, electronic equipment and storage medium |
| CN111858075A (en) * | 2020-06-30 | 2020-10-30 | 广东浪潮大数据研究有限公司 | Deadlock detection method, system, equipment and readable storage medium |
| CN112256196A (en) * | 2020-10-20 | 2021-01-22 | 北京字跳网络技术有限公司 | File reading method and device, electronic equipment and computer readable medium |
| CN112256196B (en) * | 2020-10-20 | 2023-03-24 | 北京字跳网络技术有限公司 | File reading method and device, electronic equipment and computer readable medium |
| WO2022199400A1 (en) * | 2021-03-23 | 2022-09-29 | 中兴通讯股份有限公司 | Method and apparatus for retrieving persistent memory file system metadata, and storage structure |
| CN113420024A (en) * | 2021-06-02 | 2021-09-21 | 浪潮软件股份有限公司 | Business form data sub-table storage method |
| CN113625968A (en) * | 2021-08-12 | 2021-11-09 | 网易（杭州）网络有限公司 | File authority management method and device, computer equipment and storage medium |
| CN113625968B (en) * | 2021-08-12 | 2024-03-01 | 网易（杭州）网络有限公司 | File authority management method and device, computer equipment and storage medium |
| CN114442937B (en) * | 2021-12-31 | 2023-03-28 | 北京云宽志业网络技术有限公司 | File caching method and device, computer equipment and storage medium |
| CN114442937A (en) * | 2021-12-31 | 2022-05-06 | 北京云宽志业网络技术有限公司 | File caching method and device, computer equipment and storage medium |
| CN114528258B (en) * | 2022-02-18 | 2022-12-27 | 北京百度网讯科技有限公司 | Asynchronous file processing method, device, server, medium, product and system |
| CN114528258A (en) * | 2022-02-18 | 2022-05-24 | 北京百度网讯科技有限公司 | Asynchronous file processing method, device, server, medium, product and system |
| CN117493276A (en) * | 2024-01-03 | 2024-02-02 | 柏科数据技术（深圳）股份有限公司 | Reading method and device for Ceph file, server and storage medium |
| CN117493276B (en) * | 2024-01-03 | 2024-04-09 | 柏科数据技术（深圳）股份有限公司 | Reading method and device for Ceph file, server and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109446160A (en) | A kind of file reading, system, device and computer readable storage medium | |
| KR102133951B1 (en) | Short link handling methods, devices, and servers | |
| WO2021042923A1 (en) | Data sharding storage method and apparatus, and terminal and computer-readable storage mediium | |
| EP3236365A1 (en) | Data query method and device | |
| CN105573667A (en) | Data reading method and storage server | |
| CN103077199B (en) | A kind of file resource Search and Orientation method and device | |
| CN106874348B (en) | File storage and index method and device and file reading method | |
| CN108491549B (en) | Method and device for creating file directory in distributed storage system | |
| CN105718455A (en) | Data query method and apparatus | |
| US9128944B2 (en) | File system enabling fast purges and file access | |
| CN105550371A (en) | Big data environment oriented metadata organization method and system | |
| CN104348859B (en) | File synchronisation method, device, server, terminal and system | |
| CN111324665B (en) | Log playback method and device | |
| CN107085546B (en) | Data management method and device based on fault domain technology | |
| CN105338073A (en) | File directory processing method, server and system | |
| CN110245145A (en) | Structure synchronization method and apparatus of the relevant database to Hadoop database | |
| CN103077208A (en) | Uniform resource locator matching processing method and device | |
| JP6111337B2 (en) | Database management system and database management method | |
| CN102523301A (en) | Method for caching data on client in cloud storage | |
| CN107181773A (en) | Data storage and data managing method, the equipment of distributed memory system | |
| CN109756484B (en) | Object storage-based gateway control method, control device, gateway and medium | |
| CN107423425A (en) | A kind of data quick storage and querying method to K/V forms | |
| EP3633519A1 (en) | Method for storing objects, and object store gateway | |
| CN109783462A (en) | A kind of data access method and device based on distributed file system | |
| CN103544254B (en) | A kind of data managing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190308 |
|
| RJ01 | Rejection of invention patent application after publication |