CN108897859A - A kind of metadata retrieval method, apparatus, equipment and computer readable storage medium - Google Patents
A kind of metadata retrieval method, apparatus, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN108897859A CN108897859A CN201810699153.8A CN201810699153A CN108897859A CN 108897859 A CN108897859 A CN 108897859A CN 201810699153 A CN201810699153 A CN 201810699153A CN 108897859 A CN108897859 A CN 108897859A
- Authority
- CN
- China
- Prior art keywords
- metadata
- file
- retrieval
- condition
- search condition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of metadata retrieval method, apparatus, equipment and computer readable storage medium, wherein this method includes：File storage request is received, the file storage request includes the file data and metadata of file；The file data is stored to the storage pool of storage system, the key-value pair being made of the file identification of the file and the metadata is stored into the metadata pond of the storage system；File access request is received, the search condition carried in the file access request is obtained；The metadata for meeting the search condition by retrieving in the metadata pond, based on the corresponding file identification of metadata retrieved by obtaining respective file data in the storage pool and returning.As it can be seen that technical solution disclosed in the present application can quickly be navigated to corresponding file data and be returned based on the condition that user provides, the usage experience of user is improved.
Description
Technical field
The present invention relates to cloud computation data center technical fields, more specifically to a kind of metadata retrieval method, dress
It sets, equipment and computer readable storage medium.
Background technique
With the continuous development of computer technology, the digitlization of information in physical world, social networks and e-commerce
Prevalence, data volume are in explosive growth, it was reported that, the growth rate of business data is annual 100%, then the amount of storage of data
It has been not only to be calculated with GB, TB, but gradually risen to PB even EB rank.In face of mass data so and increasing
Long trend, mass storage system (MSS) come into being, but user accesses data needs clearly to know data in storage NameSpace
Specific location, file is then obtained by specific location.This method accurately searched limits the use condition of user, nothing
Method is quickly positioned from mass file storage system based on the search condition that user provides and user is allowed directly to access text
The function of part.
It can not be based on the condition that user provides in conclusion existing in the data search technical solution realized in the prior art
The problem of quickly navigating to corresponding data.
Summary of the invention
The object of the present invention is to provide a kind of metadata retrieval method, apparatus, equipment and computer readable storage medium, with
It solves quickly navigate to based on the condition that user provides present in the data search technical solution realized in the prior art
The problem of corresponding data.
To achieve the goals above, the present invention provides the following technical solutions：
A kind of metadata retrieval method, including：
File storage request is received, the file storage request includes the file data and metadata of file；By the text
Number of packages deposits the key-value pair being made of the file identification of the file and the metadata according to storing to the storage pool of storage system
Storage is into the metadata pond of the storage system；
File access request is received, the search condition carried in the file access request is obtained；By the metadata pond
It is middle to retrieve the metadata for meeting the search condition, based on the corresponding file identification of metadata retrieved by the storage pool
It obtains respective file data and returns.
Preferably, the key-value pair being made of the file identification of the file and the metadata is stored to the storage and is
In the metadata pond of system, including：
Determine object corresponding with the type of the metadata in the metadata pond, and will be by the files-designated of the file
Know and the key-value pair of metadata composition is stored into the object determined；
It is corresponding, the metadata of the search condition is met by retrieving in the metadata pond, including：
It determines the corresponding metadata type of the search condition, and is examined in the corresponding object of the metadata type by determining
The metadata of search condition described in Suo Fuhe.
Preferably, object corresponding with the type of the metadata in the metadata pond is determined, including：
If the metadata is preset basic metadata, by pre-establishing and each preset basic metadata
The corresponding object of type in determine object corresponding with the metadata.
Preferably, object corresponding with the type of the metadata in the metadata pond is determined, including：
If the metadata is self-defining metadata, judge in the metadata pond whether include and the metadata
The corresponding object of type, determine include in the metadata pond corresponding with the type of the metadata if it is, executing
Object the step of, if it is not, then corresponding with the type of the metadata object of creation, and determine the object that creates for institute
State the corresponding object of type of metadata.
Preferably, the corresponding metadata type of the search condition is determined, and the metadata type by determining is corresponding
Retrieval meets the metadata of the search condition in object, including：
If the search condition include it is multiple retrieval sub- condition and multiple sub- search condition between for relationship,
Determine that any of the multiple sub- condition of the retrieval sub- condition of retrieval is goal condition, by member corresponding with the goal condition
Retrieval meets the metadata of the goal condition in the corresponding object of data type, and determination is corresponding with the metadata retrieved
File identification is target identification, and target identification is corresponded to key-value pair and is stored into retrieval file；Successively by the multiple retrieval
Other in condition in addition to the goal condition are all retrieved in the corresponding object of the corresponding metadata type of sub- condition, by institute
It states and does not meet the corresponding subconditional key-value pair of retrieval in the corresponding metadata of target identification by deleting in the retrieval file；It determines
It is to meet the metadata of the search condition that key-value pair in the retrieval file, which corresponds to metadata,.
Preferably, the corresponding metadata type of the search condition is determined, and the metadata type by determining is corresponding
Retrieval meets the metadata of the search condition in object, including：
If the search condition includes between the sub- condition of multiple retrievals and multiple sub- condition of retrieval be or relationship,
Retrieval in the corresponding object of metadata type is corresponded to by each sub- condition of retrieval and meets the corresponding subconditional metadata of retrieval, and really
Surely the metadata retrieved is to meet the metadata of the search condition.
Preferably, further include：
If meeting the metadata of the search condition by not retrieving in the metadata pond, returns and do not find symbol
Close the information of the metadata of search condition.
A kind of metadata retrieval device, including：
Memory module is used for：File storage request is received, the file storage request includes the file data and member of file
Data；The file data is stored to the storage pool of storage system, it will be by the file identification of the file and the metadata
The key-value pair of composition is stored into the metadata pond of the storage system；
Retrieval module is used for：File access request is received, the search condition carried in the file access request is obtained；
The metadata for meeting the search condition by retrieving in the metadata pond, based on the corresponding file identification of metadata retrieved
By obtaining respective file data in the storage pool and returning.
A kind of metadata retrieval equipment, including：
Memory, for storing computer program；
Processor realizes the step of the as above any one metadata retrieval method when for executing the computer program
Suddenly.
A kind of computer readable storage medium is stored with computer program on the computer readable storage medium, described
The step of as above any one metadata retrieval method is realized when computer program is executed by processor.
The present invention provides a kind of metadata retrieval method, apparatus, equipment and computer readable storage medium, the wherein party
Method includes：File storage request is received, the file storage request includes the file data and metadata of file；By the file
Data are stored to the storage pool of storage system, and the key-value pair being made of the file identification of the file and the metadata is stored
Into the metadata pond of the storage system；File access request is received, the retrieval carried in the file access request is obtained
Condition；The metadata for meeting the search condition by retrieving in the metadata pond, based on the corresponding text of metadata retrieved
Part mark is by obtaining respective file data and returning in the storage pool.In technical solution disclosed by the invention, file is being realized
When storage, file data is directly stored to the storage pool of storage system, and by metadata by file identification and metadata structure
At the mode of key-value pair store to the metadata pond of storage system, thus, it is possible to the search conditions provided based on user by first number
According to retrieving corresponding metadata in pond, and then return to based on the metadata retrieved the number of files of the file of respective file mark
According to；As it can be seen that technical solution disclosed in the present application can quickly navigate to corresponding file data simultaneously based on the condition that user provides
It returns, improves the usage experience of user.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of invention for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of flow chart of metadata retrieval method provided in an embodiment of the present invention；
Fig. 2 is metadata point in a kind of metadata retrieval method provided in an embodiment of the present invention
The schematic diagram of class storage；
Fig. 3 is a kind of structural representation of metadata retrieval device provided in an embodiment of the present invention
Figure.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Referring to Fig. 1, can wrap it illustrates a kind of flow chart of metadata retrieval method provided in an embodiment of the present invention
It includes：
S11：File storage request is received, file storage request includes the file data and metadata of file；By number of files
According to storing to the storage pool of storage system, the key-value pair being made of the file identification of file and metadata is stored to storage system
Metadata pond in.
A kind of execution subject of metadata retrieval method provided in an embodiment of the present invention can be corresponding device, and the dress
Setting can be set within the storage system, and therefore, a kind of execution subject of metadata retrieval method provided in an embodiment of the present invention can
Think storage system.After receiving file storage request, the file data that file storage request includes is stored to storage pool, specifically
For, it can be and file data is stored after the operation such as data slicer and data redundancy duplication into magnetic disk media.It will be first
Data are stored in a manner of the key-value pair that file identification and metadata are constituted in metadata pond, and wherein file identification is file
Unique identification, i.e., the file identification that each file has are all different, and determine corresponding file with uniqueness；And key-value pair and
KV pairs, key is file identification, and value is metadata, and key-value pair can store in the omap data of corresponding objects.
S12：File access request is received, the search condition carried in file access request is obtained；By being examined in metadata pond
The metadata of Suo Fuhe search condition, based on the corresponding file identification of metadata retrieved by obtaining respective file in storage pool
Data simultaneously return.
If receiving file request, the search condition carried in file request is obtained, wherein search condition is specially
Condition corresponding with metadata, such as it is picture, picture size 25KB that metadata, which includes file type, then search condition can be with
For file type be picture or file type is picture and picture size is not more than 25KB, etc..Thereby, it is possible in metadata
The metadata for meeting search condition is retrieved in pond, and then is determined corresponding with the metadata composition file identification of key-value pair
The file data that the file data of file as needs to return, and by returning to use after reading respective file data in storage pool
Family.
In technical solution disclosed by the invention, when realizing file storage, file data is directly stored to storage system
Storage pool, and metadata is stored in a manner of the key-value pair being made of file identification and metadata to first number of storage system
According to pond, thus, it is possible to the search conditions provided based on user by retrieving corresponding metadata in metadata pond, and then based on inspection
The metadata that rope arrives returns to the file data of the file of respective file mark；As it can be seen that technical solution disclosed in the present application being capable of base
Corresponding file data is quickly navigated in the condition that user provides and is returned, and the usage experience of user is improved.
Also, storage pool and metadata pond are respectively positioned in storage system in technical solution disclosed in the present application, and then are based on
The retrieval of metadata and the return of respective file data are realized in storage pool and metadata pond, without third party software, significantly
Reduce the safety risks by realizing corresponding step using third party software.
It is further to note that can also include after receiving file access request：Judge in file access request
Whether search condition is carried, if it is, the step of obtaining the search condition carried in file access request is executed, if not,
Respective file is then directly targeted to based on the file identification carried in file access request, and by the number of files of the file navigated to
According to returning to user.Enabling a user to flexible choice according to actual needs is that file identification is carried on file access to ask
It asks or search condition is carried in file access request, further the user experience is improved.
A kind of metadata retrieval method provided in an embodiment of the present invention, by what is be made of the file identification of file and metadata
Key-value pair is stored into the metadata pond of storage system, may include：
Determine object corresponding with the type of metadata in metadata pond, and will be by the file identification of file and metadata structure
At key-value pair store into the object determined；
It is corresponding, the metadata of search condition is met by retrieving in metadata pond, including：
It determines the corresponding metadata type of search condition, and retrieves symbol in the corresponding object of the metadata type by determining
Close the metadata of search condition.
It should be noted that metadata can be carried out classification storage, i.e. metadata type and object when storage metadata
It corresponds, to storing metadata into object corresponding with the metadata type；Metadata carries out showing for classification storage
Intention can be as shown in Figure 2, wherein data are file data.For example, as metadata include file type be picture,
File size is 25KB, then file type is that the metadata of picture is stored into file type corresponding objects, and file size is
The metadata of 25KB is stored into file size corresponding objects；And then when realizing the retrieval of metadata, it can first determine to examine
Rope condition is which kind of corresponding metadata type, and then retrieval meets the member of search condition in the corresponding object of the metadata type
Data are picture if search condition is file type, then are picture by search file type in the corresponding object of file type
Metadata.Without the metadata of traversal, but can first determine to need based on the corresponding metadata type of search condition
It realizes the object of retrieval, and then realizes the retrieval for meeting the metadata of search condition in the object, pass through this metadata point
The mode of class storage and retrieval substantially increases the recall precision of metadata.
A kind of metadata retrieval method provided in an embodiment of the present invention determines corresponding with the type of metadata in metadata pond
Object, may include：
If metadata is preset basic metadata, by the class with each preset basic metadata pre-established
Object corresponding with metadata is determined in the corresponding object of type.
It should be noted that basic first number can be referred to as the metadata for all file stored being needed to all have
According to this is determined according to the practical metadata having of file.Such as filename, file type, file size, document creation modification
Time etc. can pre-establish and each basic first number due to all the file stored being needed to all have basic metadata
According to the corresponding object of type, so that directly basic metadata can be stored to correspondence when realizing the storage of basic metadata
Object in, substantially increase the storage efficiency of metadata.
A kind of metadata retrieval method provided in an embodiment of the present invention determines corresponding with the type of metadata in metadata pond
Object, may include：
If metadata is self-defining metadata, whether judge in metadata pond comprising corresponding with the type of metadata
Object, if it is, the step of determining the object corresponding with the type of metadata for including in metadata pond is executed, if not,
Object corresponding with the type of metadata is then created, and determines that the object of creation is object corresponding with the type of metadata.
It should be noted that in some cases, user also wants to setting self-defining metadata, as file significance level,
File storage time etc., since these metadata are the customized metadata of user, what not all file all had, therefore
If pre-establishing the object of each corresponding self-defining metadata in the way of basic metadata, it will appear waste unavoidably and deposit
The case where storing up resource；Therefore object corresponding with self-defining metadata is not pre-established in the application, but is needing to store
The creation of self-defining metadata corresponding objects and customized is just realized when the self-defining metadata of some type through the above way
The storage of metadata, to effectively prevent the waste of storage resource.
A kind of metadata retrieval method provided in an embodiment of the present invention determines the corresponding metadata type of search condition, and
Meet the metadata of search condition by retrieving in the corresponding object of the metadata type determined, may include：
If search condition include it is multiple retrieval sub- condition and multiple sub- search condition between for relationship, it is determined that
Any of multiple sub- conditions of retrieval sub- condition of retrieval is goal condition, corresponding by metadata type corresponding with goal condition
Object in retrieval meet the metadata of goal condition, and determine that file identification corresponding with the metadata retrieved is target mark
Know, target identification is corresponded into key-value pair and is stored into retrieval file；Successively retrieved in sub- conditions in addition to goal condition by multiple
Other all retrieve in the corresponding objects of the corresponding metadata type of sub- condition, will not be inconsistent in the corresponding metadata of target identification
The corresponding subconditional key-value pair of retrieval is closed by deleting in retrieval file；Determine that the key-value pair in retrieval file corresponds to metadata as symbol
Close the metadata of search condition.
Sub- search condition be with relationship then illustrate that the metadata for meeting this little search condition simultaneously is only and meet retrieval
The metadata of condition, if sub- search condition be or relationship then illustrate at least to meet the son inspection of any one of this little search condition
The metadata of rope condition is to meet the metadata of search condition, naturally it is also possible to which search condition occur both includes "AND" relationship
Metadata, the metadata for including the case where "or" relationship again, then for the wherein member of the metadata of "AND" relationship and "or" relationship
The principle of data is identical as above-mentioned principle, these be in mathematical logic in the prior art, details are not described herein.Wherein, right
The case where being "AND" between multiple sub- search conditions that search condition includes, can first determine any son by above-mentioned
Search condition respective file mark, and then realize that the mode of later retrieval is realized based on the file identification determined, relative to according to
Metadata in the secondary each corresponding objects of retrieval will greatly shorten retrieval time, improve recall precision.
A kind of metadata retrieval method provided in an embodiment of the present invention determines the corresponding metadata type of search condition, and
Meet the metadata of search condition by retrieving in the corresponding object of the metadata type determined, may include：
If search condition includes between the sub- condition of multiple retrievals and multiple sub- condition of retrieval be or relationship, by every
A sub- condition of retrieval corresponds to retrieval in the corresponding object of metadata type and meets the corresponding subconditional metadata of retrieval, and determines inspection
The metadata that rope goes out is to meet the metadata of search condition.
The case where being the relationship of "or" for search condition between the sub- condition of multiple retrievals for including, then it can exist respectively
Corresponding metadata is found in corresponding objects, it is simple easily to realize.
A kind of metadata retrieval method provided in an embodiment of the present invention can also include：
If meeting the metadata of search condition by not retrieving in metadata pond, returns not find and meet retrieval item
The information of the metadata of part.
If not finding the metadata for meeting search condition, the information for not finding qualified metadata is returned
To user, enables to user's timely learning situation and realize corresponding operation.
It is further to note that in a kind of concrete application scene, a kind of metadata retrieval method disclosed in the present application
Middle file storing process may comprise steps of：
1) user sends file storage request from application end, carries file data and metadata, wherein file metadata packet
Containing basic metadata, self-defining metadata may include.
2) by file data according to distributed objects storage agreement (storage system is distributed objects storage system) storage
To data pool, it specifically can be by data slicer and data redundancy duplication, store into magnetic disk media.
3) metadata storage to metadata pond, specifically, basic metadata can according to basic metadata type into
Row classification storage, such as file test.jpg, file identification (oid) is test-oid, and file type is picture, and file size is
25KB, then by " test-oid:Test.jpg " is inserted into the omap KV key-value pair of the object of " file name "；By " test-
oid:Picture " is inserted into the omap KV key-value pair of the object of " file type "；By " test-oid:25KB " is inserted into
In the omap KV key-value pair of the object of " file size ".
4) file metadata storage specifically, can first judge whether there is self-defining metadata, such as to metadata pond
Fruit is not present, and jumps to 7)；If it is present judge the object of the whether existing identical type of self-defining metadata, if
In the presence of jumping to 5)；If it does not, jumping to 6).
5) self-defining metadata is then inserted by self-defining metadata there are the object of identical type in metadata pond
In the omap KV key-value pair of this object, such as the self-defining metadata of file test.jpg be image content is animal painting, and
There is the type object of " image content classification " in metadata pond, then by " test-oid:Animal " is inserted into " image content
In the omap KV key-value pair of classification " object.
6) object of user's self-defining metadata without identical type in metadata pond, then new Object, and will use
Family self-defining metadata is saved in the omap KV key-value pair of this object, such as the customized former data of file test.jpg are
Picture color is blue, and the object of " picture color " is not present in metadata pond, then " picture face is created in metadata pond
The object of color ", and by " test-oid:Blue " is saved in the omap KV key-value pair of " picture color " object.
7) file data and metadata save success, then entire file stores successfully；Otherwise file storage failure.If
File storage failure can return to the information of file storage failure to user.
Corresponding above-mentioned application scenarios, metadata retrieval process can wrap in a kind of metadata retrieval method disclosed in the present application
Include following steps：
(1) user sends file access request from application end.
(2) judge whether carry search condition in file access request, if not, jumping to (3)；If so, jumping to
(4)。
(3) it according to the file identification of file access request, is returned to the interior file data that reads of storage pool.
(4) document retrieval is carried out according to the search condition of file access request, document retrieval condition there can be one or more
It is a, can be with or search condition, if being successfully retrieved the metadata for meeting search condition, by respective file data
It returns, otherwise then returns to the information for not finding the metadata for meeting search condition.Wherein, above-mentioned retrieving is illustrated：
Search condition is the picture file greater than 10KB, then inquiry is greater than 10KB in the object omap KV key-value pair of " file size "
File identification, the file identification that front is got then is found in the object omap KV key-value pair of " file type ", if
Its value is picture, then is retrieved the final result as this retrieval, and the file identification for returning to final result is corresponding
File data.
As it can be seen that storage system belongs to KV key-value pair storage system in technical solution disclosed in the present application, has member abundant
Data information has metadata retrieval function；Independent of external other assemblies, metadata storage and retrieval logic are being deposited completely
It is realized inside storage system；It can flexibly use, be confirmed whether to carry out metadata inspection according to the required parameter that user's application end is sent
Rope promotes request and returns to efficiency.
The embodiment of the invention also provides a kind of metadata retrieval devices, as shown in figure 3, may include：
Memory module 11, is used for：File storage request is received, file storage request includes the file data and first number of file
According to；File data is stored to the storage pool of storage system, the key-value pair being made of the file identification of file and metadata is deposited
Storage is into the metadata pond of storage system；
Retrieval module 12, is used for：File access request is received, the search condition carried in file access request is obtained；By
Retrieval meets the metadata of search condition in metadata pond, based on the corresponding file identification of metadata retrieved by storage pool
It obtains respective file data and returns.
A kind of metadata retrieval device provided in an embodiment of the present invention, memory module may include：
Storage unit is used for：Determine object corresponding with the type of metadata in metadata pond, and will be by the file of file
The key-value pair that mark and metadata are constituted is stored into the object determined；
Corresponding, retrieval module may include：
Retrieval unit is used for：Determine the corresponding metadata type of search condition, and the metadata type by determining is corresponding
Object in retrieval meet the metadata of search condition.
A kind of metadata retrieval device provided in an embodiment of the present invention, storage unit may include：
Second storing sub-units, are used for：If metadata be preset basic metadata, by it is pre-establishing and each
Object corresponding with metadata is determined in the corresponding object of type of preset basic metadata.
A kind of metadata retrieval device provided in an embodiment of the present invention, storage unit may include：
Second storing sub-units, are used for：If metadata is self-defining metadata, judge in metadata pond whether include
Object corresponding with the type of metadata, if it is, include in the determining metadata pond of execution is corresponding with the type of metadata
Object the step of, if it is not, then corresponding with the type of the metadata object of creation, and determine that the object created is and metadata
The corresponding object of type.
A kind of metadata retrieval device provided in an embodiment of the present invention, retrieval unit may include：
First retrieval subelement, is used for：If search condition includes the sub- condition of multiple retrievals and multiple sub- search condition
Between for relationship, it is determined that the sub- conditions of any of sub- conditions of multiple retrievals retrieval are goal condition, by with target item
Retrieval meets the metadata of goal condition, and the metadata for determining and retrieving in the corresponding object of the corresponding metadata type of part
Corresponding file identification is target identification, and target identification is corresponded to key-value pair and is stored into retrieval file；Successively by multiple retrievals
Other in sub- condition in addition to goal condition are all retrieved in the corresponding object of the corresponding metadata type of sub- condition, by target
It identifies and does not meet the corresponding subconditional key-value pair of retrieval in corresponding metadata by deleting in retrieval file；It determines in retrieval file
Key-value pair to correspond to metadata be to meet the metadata of search condition.
A kind of metadata retrieval device provided in an embodiment of the present invention, retrieval unit may include：
Second retrieval subelement, is used for：If search condition includes the sub- condition of multiple retrievals and multiple sub- condition of retrieval
Between be or relationship, then correspond to retrieval in the corresponding object of metadata type by each retrieval sub- condition, to meet corresponding retrieval sub
The metadata of condition, and determine that the metadata retrieved is to meet the metadata of search condition.
A kind of metadata retrieval device provided in an embodiment of the present invention can also include：
Return module is used for：If meeting the metadata of search condition by not retrieving in metadata pond, returns and do not look into
Find the information for meeting the metadata of search condition.
The embodiment of the invention also provides a kind of metadata retrieval equipment, may include：
Memory, for storing computer program；
Processor, when for executing computer program realize as above any one of metadata retrieval method the step of.
The embodiment of the invention also provides a kind of computer readable storage medium, it is stored on computer readable storage medium
Computer program, when computer program is executed by processor realize as above any one of metadata retrieval method the step of.
It should be noted that a kind of metadata retrieval device provided in an embodiment of the present invention, equipment and computer-readable depositing
The explanation of relevant portion refers to corresponding part in a kind of metadata retrieval method provided in an embodiment of the present invention in storage media
It is described in detail, details are not described herein.In addition in above-mentioned technical proposal provided in an embodiment of the present invention with correspond to skill in the prior art
The consistent part of art scheme realization principle is simultaneously unspecified, in order to avoid excessively repeat.
The foregoing description of the disclosed embodiments can be realized those skilled in the art or using the present invention.To this
A variety of modifications of a little embodiments will be apparent for a person skilled in the art, and the general principles defined herein can
Without departing from the spirit or scope of the present invention, to realize in other embodiments.Therefore, the present invention will not be limited
It is formed on the embodiments shown herein, and is to fit to consistent with the principles and novel features disclosed in this article widest
Range.
Claims (10)
1. a kind of metadata retrieval method, which is characterized in that including：
File storage request is received, the file storage request includes the file data and metadata of file；By the number of files
According to storing to the storage pool of storage system, by the key-value pair being made of the file identification of the file and the metadata store to
In the metadata pond of the storage system；
File access request is received, the search condition carried in the file access request is obtained；By being examined in the metadata pond
The metadata of search condition described in Suo Fuhe, based on the corresponding file identification of metadata retrieved by being obtained in the storage pool
Respective file data simultaneously return.
2. the method according to claim 1, wherein by by the file identification of the file and the metadata structure
At key-value pair store into the metadata pond of the storage system, including：
Determine object corresponding with the type of the metadata in the metadata pond, and by by the file identification of the file and
The key-value pair that the metadata is constituted is stored into the object determined；
It is corresponding, the metadata of the search condition is met by retrieving in the metadata pond, including：
It determines the corresponding metadata type of the search condition, and retrieves symbol in the corresponding object of the metadata type by determining
Close the metadata of the search condition.
3. according to the method described in claim 2, it is characterized in that, determining the type in the metadata pond with the metadata
Corresponding object, including：
If the metadata is preset basic metadata, by the class with each preset basic metadata pre-established
Object corresponding with the metadata is determined in the corresponding object of type.
4. according to the method described in claim 3, it is characterized in that, determining the type in the metadata pond with the metadata
Corresponding object, including：
If the metadata is self-defining metadata, whether judge in the metadata pond comprising the class with the metadata
The corresponding object of type, if it is, include in the determining metadata pond of execution is corresponding with the type of the metadata right
As the step of, if it is not, then corresponding with the type of the metadata object of creation, and determine the object of creation for and the member
The corresponding object of the type of data.
5. according to the method described in claim 2, it is characterized in that, determine the corresponding metadata type of the search condition, and
Meet the metadata of the search condition by retrieving in the corresponding object of the metadata type determined, including：
If the search condition include it is multiple retrieval sub- condition and multiple sub- search condition between for relationship, it is determined that
Any of the multiple sub- condition of the retrieval sub- condition of retrieval is goal condition, by metadata corresponding with the goal condition
Retrieval meets the metadata of the goal condition in the corresponding object of type, and determines file corresponding with the metadata retrieved
It is identified as target identification, target identification is corresponded into key-value pair and is stored into retrieval file；Successively by the multiple sub- condition of retrieval
In other in addition to the goal condition all retrieve in the corresponding objects of the corresponding metadata type of sub- condition, by the mesh
Mark, which identifies, does not meet the corresponding subconditional key-value pair of retrieval by deleting in the retrieval file in corresponding metadata；Described in determination
It is to meet the metadata of the search condition that key-value pair in retrieval file, which corresponds to metadata,.
6. according to the method described in claim 5, it is characterized in that, determine the corresponding metadata type of the search condition, and
Meet the metadata of the search condition by retrieving in the corresponding object of the metadata type determined, including：
If the search condition includes between the sub- condition of multiple retrievals and multiple sub- condition of retrieval be or relationship, by every
A sub- condition of retrieval corresponds to retrieval in the corresponding object of metadata type and meets the corresponding subconditional metadata of retrieval, and determines inspection
The metadata that rope goes out is to meet the metadata of the search condition.
7. the method according to claim 1, wherein further including：
If meeting the metadata of the search condition by not retrieving in the metadata pond, returns not find and meet inspection
The information of the metadata of rope condition.
8. a kind of metadata retrieval device, which is characterized in that including：
Memory module is used for：File storage request is received, the file storage request includes the file data and first number of file
According to；The file data is stored to the storage pool of storage system, it will be by the file identification of the file and the metadata structure
At key-value pair store into the metadata pond of the storage system；
Retrieval module is used for：File access request is received, the search condition carried in the file access request is obtained；By institute
The metadata that retrieval in metadata pond meets the search condition is stated, based on the corresponding file identification of metadata retrieved by institute
It states and obtains respective file data in storage pool and return.
9. a kind of metadata retrieval equipment, which is characterized in that including：
Memory, for storing computer program；
Processor realizes the metadata retrieval method as described in any one of claim 1 to 7 when for executing the computer program
The step of.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program, realizing the metadata retrieval method as described in any one of claim 1 to 7 when the computer program is executed by processor
Step.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810699153.8A CN108897859A (en) | 2018-06-29 | 2018-06-29 | A kind of metadata retrieval method, apparatus, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810699153.8A CN108897859A (en) | 2018-06-29 | 2018-06-29 | A kind of metadata retrieval method, apparatus, equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108897859A true CN108897859A (en) | 2018-11-27 |
Family
ID=64347368
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810699153.8A Pending CN108897859A (en) | 2018-06-29 | 2018-06-29 | A kind of metadata retrieval method, apparatus, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108897859A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109918234A (en) * | 2019-03-06 | 2019-06-21 | 苏州浪潮智能科技有限公司 | A kind of metadata restoration methods, device, equipment and medium based on SSD |
| CN111104366A (en) * | 2019-12-04 | 2020-05-05 | 苏州智加科技有限公司 | Data archiving method, device, server and storage medium |
| CN111752954A (en) * | 2020-06-29 | 2020-10-09 | 深圳前海微众银行股份有限公司 | Large-scale feature data storage method and device |
| CN111984691A (en) * | 2020-09-11 | 2020-11-24 | 苏州浪潮智能科技有限公司 | Object metadata retrieval and enumeration method and device in distributed storage system |
| CN113127420A (en) * | 2021-03-30 | 2021-07-16 | 山东英信计算机技术有限公司 | Metadata request processing method, device, equipment and medium |
| CN113312355A (en) * | 2021-06-15 | 2021-08-27 | 北京沃东天骏信息技术有限公司 | Data management method and device |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1996281A (en) * | 2005-12-26 | 2007-07-11 | 北大方正集团有限公司 | Method for building content management system |
| CN101980203A (en) * | 2010-10-29 | 2011-02-23 | 中国科学院声学研究所 | Embedded file system for high-definition media |
| CN105630919A (en) * | 2015-12-22 | 2016-06-01 | 曙光信息产业(北京)有限公司 | Storage method and system |
| CN105677826A (en) * | 2016-01-04 | 2016-06-15 | 博康智能网络科技股份有限公司 | Resource management method for massive unstructured data |
| CN105678189A (en) * | 2016-01-15 | 2016-06-15 | 上海海事大学 | Encrypted data file storage and retrieval system and method |
| CN105808622A (en) * | 2014-12-31 | 2016-07-27 | 乐视网信息技术（北京）股份有限公司 | File storage method and device |
| US20170060941A1 (en) * | 2015-08-25 | 2017-03-02 | Futurewei Technologies, Inc. | Systems and Methods for Searching Heterogeneous Indexes of Metadata and Tags in File Systems |
| CN106970958A (en) * | 2017-03-17 | 2017-07-21 | 浙江大华技术股份有限公司 | A kind of inquiry of stream file and storage method and device |
| CN107844546A (en) * | 2017-10-27 | 2018-03-27 | 郑州云海信息技术有限公司 | A kind of file system metadata management system and method |
-
2018
- 2018-06-29 CN CN201810699153.8A patent/CN108897859A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1996281A (en) * | 2005-12-26 | 2007-07-11 | 北大方正集团有限公司 | Method for building content management system |
| CN101980203A (en) * | 2010-10-29 | 2011-02-23 | 中国科学院声学研究所 | Embedded file system for high-definition media |
| CN105808622A (en) * | 2014-12-31 | 2016-07-27 | 乐视网信息技术（北京）股份有限公司 | File storage method and device |
| US20170060941A1 (en) * | 2015-08-25 | 2017-03-02 | Futurewei Technologies, Inc. | Systems and Methods for Searching Heterogeneous Indexes of Metadata and Tags in File Systems |
| CN105630919A (en) * | 2015-12-22 | 2016-06-01 | 曙光信息产业(北京)有限公司 | Storage method and system |
| CN105677826A (en) * | 2016-01-04 | 2016-06-15 | 博康智能网络科技股份有限公司 | Resource management method for massive unstructured data |
| CN105678189A (en) * | 2016-01-15 | 2016-06-15 | 上海海事大学 | Encrypted data file storage and retrieval system and method |
| CN106970958A (en) * | 2017-03-17 | 2017-07-21 | 浙江大华技术股份有限公司 | A kind of inquiry of stream file and storage method and device |
| CN107844546A (en) * | 2017-10-27 | 2018-03-27 | 郑州云海信息技术有限公司 | A kind of file system metadata management system and method |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109918234A (en) * | 2019-03-06 | 2019-06-21 | 苏州浪潮智能科技有限公司 | A kind of metadata restoration methods, device, equipment and medium based on SSD |
| CN111104366A (en) * | 2019-12-04 | 2020-05-05 | 苏州智加科技有限公司 | Data archiving method, device, server and storage medium |
| CN111104366B (en) * | 2019-12-04 | 2024-01-19 | 苏州智加科技有限公司 | Data archiving method, device, server and storage medium |
| CN111752954A (en) * | 2020-06-29 | 2020-10-09 | 深圳前海微众银行股份有限公司 | Large-scale feature data storage method and device |
| CN111984691A (en) * | 2020-09-11 | 2020-11-24 | 苏州浪潮智能科技有限公司 | Object metadata retrieval and enumeration method and device in distributed storage system |
| CN111984691B (en) * | 2020-09-11 | 2023-01-06 | 苏州浪潮智能科技有限公司 | Object metadata retrieval and enumeration method and device in distributed storage system |
| US12013861B2 (en) | 2020-09-11 | 2024-06-18 | Inspur Suzhou Intelligent Technology Co., Ltd. | Method and apparatus for retrieving and enumerating object metadata in distributed storage system |
| CN113127420A (en) * | 2021-03-30 | 2021-07-16 | 山东英信计算机技术有限公司 | Metadata request processing method, device, equipment and medium |
| CN113127420B (en) * | 2021-03-30 | 2023-03-14 | 山东英信计算机技术有限公司 | Metadata request processing method, device, equipment and medium |
| CN113312355A (en) * | 2021-06-15 | 2021-08-27 | 北京沃东天骏信息技术有限公司 | Data management method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108897859A (en) | A kind of metadata retrieval method, apparatus, equipment and computer readable storage medium | |
| CN108664650B (en) | Transaction processing method, device, equipment and storage medium of block chain network | |
| CN103150394B (en) | Distributed file system metadata management method facing to high-performance calculation | |
| CN109117275B (en) | Account checking method and device based on data slicing, computer equipment and storage medium | |
| CN103312624B (en) | A kind of Message Queuing Services system and method | |
| CN108647151A (en) | It is a kind of to dodge system metadata rule method, apparatus, equipment and storage medium entirely | |
| CN103559231B (en) | A kind of file system quota management method, apparatus and system | |
| CN105701096A (en) | Index generation method, data inquiry method, index generation device, data inquiry device and system | |
| WO2018121430A1 (en) | File storage and indexing method, apparatus, media, device and method for reading files | |
| CN103795811B (en) | Information storage and data statistical management method based on meta data storage | |
| CN109542857B (en) | Audit log storage method, audit log query method, audit log storage device, audit log query device and related equipment | |
| CN110471750B (en) | Timing task control method, device, medium and computer equipment | |
| CN105677904B (en) | Small documents storage method and device based on distributed file system | |
| CN104021223B (en) | The access method and device of a kind of Cluster Database measuring point | |
| CN110399348A (en) | File deletes method, apparatus, system and computer readable storage medium again | |
| CN108268216B (en) | Data processing method, device and server | |
| CN110287201A (en) | Data access method, device, equipment and storage medium | |
| CN106686095A (en) | Data storage method and device based on erasure code technology | |
| CN105869057A (en) | Comment storage device, comment reading method and apparatus, and comment writing method and apparatus | |
| CN102542036B (en) | Use the table optimized | |
| CN110008197A (en) | A kind of data processing method, system and electronic equipment and storage medium | |
| CN108154024A (en) | A kind of data retrieval method, device and electronic equipment | |
| EP4002110A1 (en) | System and method for adaptive release of application program in different environments | |
| CN102724301B (en) | Cloud database system and method and equipment for reading and writing cloud data | |
| CN111752941B (en) | Data storage and access method and device, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181127 |