CN110413807A - A kind of image inquiry method and system based on contents semantic metadata - Google Patents

A kind of image inquiry method and system based on contents semantic metadata Download PDF

Info

Publication number
CN110413807A
CN110413807A CN201910546661.7A CN201910546661A CN110413807A CN 110413807 A CN110413807 A CN 110413807A CN 201910546661 A CN201910546661 A CN 201910546661A CN 110413807 A CN110413807 A CN 110413807A
Authority
CN
China
Prior art keywords
hash
image
semantic
hamming
node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910546661.7A
Other languages
Chinese (zh)
Other versions
CN110413807B (en
Inventor
周可
刘毅斐
刘渝
汪洋涛
杨玉娟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN201910546661.7A priority Critical patent/CN110413807B/en
Publication of CN110413807A publication Critical patent/CN110413807A/en
Application granted granted Critical
Publication of CN110413807B publication Critical patent/CN110413807B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/51—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Library & Information Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a kind of image inquiry methods based on contents semantic metadata, comprising: the image file of upload is input in depth self study Hash network, handles to obtain the corresponding semantic Hash codes of image by depth self study Hash；Semantic Hash codes and image file are uploaded in storage system correspondingly, and using this Hash codes as the semantic metadata of respective image file；A both ends connection Hash table is arranged to be chained up the image file with identical semantic cryptographic Hash with chained list, chained list has the image file of identical semantic cryptographic Hash for tissue；Semantic query request is initiated by semantic query interface, semantic query interface obtains the semantic Hash codes in semantic query request by Hamming figure；Hash table is connected by both ends and initiates image file request corresponding with semantic Hash codes to the storage system, and returns to image file corresponding with semantic Hash codes.Realize the image file inquiry based on semantic content.

Description

A kind of image inquiry method and system based on contents semantic metadata
Technical field
The invention belongs to data retrieval technology fields, more particularly, to a kind of image based on contents semantic metadata Querying method and system.
Background technique
In data-storage system, metadata has proven to vital part in storage system.The study found that Although what metadata only accounted for storage system capacity is no more than 1%, have more than in storage system 50% operation need using Metadata.
When user needs to inquire file content itself, only rely on existing metadata structure be it is invalid, because Information relevant to file content is not stored for these metadata.Such as the scene in a large-scale image storage system In, user wants to find out image (image as being cat) similar with a specific image content, in this case simply Metasearch is helpless；Because of simple metadata (such as file size, creation time etc.) and text in storage system The content of part is not associated with, and in order to solve this problem, researchers propose can query semantics storage system.
Can query semantics storage system refer to inquiry operation can be supported according to the building of semantic in system and relevance Storage system.At present mainstream can be there are three types of voice inquirement storage systems, the first is looking in the distance of proposing of Leung A W et al. The same or similar file of NameSpace is carried out tissue by K-D tree, to accelerate first number by mirror (Spyglass) storage system According to inquiry；It is for second intelligent storage (SmartStore) system that Hua Y et al. is proposed, passes through latent semantic analysis The similar file of metadata is clustered, accelerates the process of inquiry by R tree；The third is the FAST that Hua Y et al. is proposed System, the scale invariant feature for extracting image in storage system convert (Scale-invariant feature Transform, abbreviation SIFT) feature, these features are grouped with local sensitivity Hash, so that its part of similar feature Sensitive hash value can be more nearly.
Then, it is above-mentioned can query semantics storage system all have the defects that some can not ignore: firstly, first two system benefit It is the unrelated conventional metadata of semantic content, can not solves the problems, such as the semantic query based on content；Secondly, the third system Only accuracy is supported to search, does not support similarity search, that is, cannot achieve complex query (such as range query)；Finally, due to The third system has used SIFT feature, is the feature of manual extraction rather than the semantic feature of image file profound level, because This third system only supports the content search based on data, but can not support semantic-based content search.
Summary of the invention
Aiming at the above defects or improvement requirements of the prior art, the present invention provides a kind of based on contents semantic metadata Image inquiry method and system, it is intended that the contents semantic information of file itself to be fused to the metadata of storage system In structure so that storage system support the semantic query based on content, thus solve it is existing can query semantics storage system Present in can not be based on semantic the technical issues of carrying out file polling of content, and the present invention can support similitude to look into It looks for, so as to realize complex query.
To achieve the above object, according to one aspect of the present invention, a kind of figure based on contents semantic metadata is provided As querying method, comprising the following steps:
(1) from image data concentrate obtain image collection, and using learning-oriented hash algorithm to the image collection at Reason, to obtain the Hash label of the image collection；
(2) using image collection as the input of neural network, the Hash label for the image collection that step (1) is obtained is made For the output of neural network, training is iterated to neural network, to obtain trained neural network；
(3) new image collection is obtained from image data set, by image collection input step (2) the trained nerve In network, to obtain the cryptographic Hash of the image collection, the cryptographic Hash of each image in image collection is stored in correspondence image In metadata, and closed using the mapping between the corresponding image of each cryptographic Hash that separation link Hash table indicates System；
(4) all cryptographic Hash in separation link Hash table used according to step (3) construct Hamming figure；
(5) semantic query received from user terminal is requested, and is parsed to semantic query request, to be checked to obtain Image is ask, corresponding cryptographic Hash is obtained in the metadata of the image to be checked；
(6) corresponding node, root are searched in the Hamming figure that step (4) is established according to the cryptographic Hash obtained in step (5) It is investigated that the node and combination Hamming figure that find obtain other all nodes for meeting semantic query request and its corresponding cryptographic Hash, According to separation link Hash table determine the corresponding cryptographic Hash of other all nodes obtained corresponding to image as final inquiry As a result.
Preferably, learning-oriented hash algorithm is depth self study hash algorithm, and image data set is CIFAR-10, STL- 10 or ImageNet.
Preferably, neural network used in step (2) is SimpleNet neural network.
Preferably, the process for executing repetitive exercise is as follows, initial parameter is arranged for neural network first, then by image set It closes and inputs in the neural network, to be exported as a result, the Hash label that the output result is obtained with step (1) is compared Compared with, then by back-propagation algorithm update neural network initial parameter, be then iteratively repeated the above process, until output tie Until error between fruit and Hash label reaches preset threshold.
Preferably, step (4) specifically includes following sub-step:
Counter i=1 is arranged in (4-1)；
(4-2) takes out i-th of cryptographic Hash in separation link Hash table, and it is corresponding that i-th cryptographic Hash is established in Hamming figure Node, and the Hamming distance between i-th of cryptographic Hash and each cryptographic Hash having been taken out before this is calculated, from calculating Select minimum value in obtained multiple Hamming distances, using the corresponding all cryptographic Hash of the minimum value as in Hamming figure with this i-th The node that the corresponding node of a cryptographic Hash is connected, between the corresponding each cryptographic Hash of the minimum value and i-th of cryptographic Hash Hamming distance is as the weight for connecting side in Hamming figure between two nodes；
(4-3) judge i whether be in separation link Hash table cryptographic Hash last, if it is process knot Otherwise i=i+1, and return step (4-2) is arranged in beam.
Preferably, its for meeting semantic query request is obtained in step (6) according to the node and combination Hamming figure that find His all nodes and its corresponding cryptographic Hash determine the corresponding Hash of all nodes of other that obtain according to separation link Hash table The corresponding image of value includes following sub-step as final query result:
(6-1) is using the node found as present node；
(6-2) determines the quantity L for all nodes being connected in Hamming figure with present node, and counter j=1 is arranged；
(6-3) judges whether counter j is less than or equal to L, is if it is transferred to step (6-4), is otherwise transferred to step (6- 8)；
(6-4) judges the Hamming distance between j-th of node being connected in Hamming figure with present node and the node found From whether Semantic Similarity threshold value is less than or equal to, if yes then enter step (6-5), else process terminates；
J-th of node is put into query result set by (6-5)；
(6-6) judges whether the Hamming distance between j-th of node and the node found is less than Semantic Similarity threshold value, If it is j-th of node is put into node set, subsequently into step (6-7), else process terminates；
Counter j=j+1, and return step (6-3) is arranged in (6-7)；
(6-8) judges whether node set is sky, if it is takes out all nodes in query result set, and according to this Cryptographic Hash inquiry separation link Hash table of a little nodes in Hamming figure, to obtain the corresponding image of all nodes as finally looking into It askes as a result, process terminates；Otherwise it is transferred to step (6-9)；
(6-9) takes out a node as present node at random from node set, by the present node from node set Middle deletion, and return step (6-2).
It is another aspect of this invention to provide that providing a kind of image query systems based on contents semantic metadata, comprising:
First module obtains image collection for concentrating from image data, and using learning-oriented hash algorithm to the image Set is handled, to obtain the Hash label of the image collection；
Second module, for using image collection as the input of neural network, the image collection that the first module is obtained Output of the Hash label as neural network, training is iterated to neural network, to obtain trained neural network；
The image collection is inputted the second module instruction for obtaining new image collection from image data set by third module In the neural network perfected, to obtain the cryptographic Hash of the image collection, the cryptographic Hash of each image in image collection is stored in In the metadata of correspondence image, and using between the corresponding image of each cryptographic Hash that separation link Hash table indicates Mapping relations；
4th module, all cryptographic Hash in separation link Hash table for being used according to third module construct Hamming Figure；
5th module, for receive from user terminal semantic query request, to the semantic query request parse, To obtain image to be checked, corresponding cryptographic Hash is obtained in the metadata of the image to be checked；
6th module, for being searched in the Hamming figure that the 4th module is established according to the cryptographic Hash obtained in the 5th module pair The node answered according to the node that finds and combines Hamming figure to obtain to meet other all nodes of semantic query request and its right The cryptographic Hash answered, according to separation link Hash table determine the corresponding cryptographic Hash of other all nodes obtained corresponding to image make For final query result.
In general, through the invention it is contemplated above technical scheme is compared with the prior art, can obtain down and show Beneficial effect:
(1) present invention is able to solve existing can not support semantic-based content to look into existing for query semantics storage system The technical issues of inquiry: since present invention employs DSTH algorithms to extract semantic similar information, and more than pixel is similar Information, so that the present invention can be suitable for semantic-based content search.
(2) since the present invention constructs Hamming figure, semantic similar node is focused into adjacent region in figure, into And the lookup of similar documents can be carried out by simple graph search mode, and do not have to the metadata progress time to All Files It goes through, thus significantly reduces the required time overhead of inquiry.
(3) since present invention employs DSTH algorithm, high efficiency makes the generation time of Hash codes of short duration, so as to Guarantee the high-performance in metadata upload procedure, is also beneficial to distribution and the storing process of metadata.
It (4), not only can be by the present invention in that multiple Hash codes merging optimization with separation link Hash table Most semantic similar image is picked out in semantic query, it can be to reduce the storage overhead and computing cost of Hamming figure significantly.
Detailed description of the invention
Fig. 1 shows semantic similar difference schematic diagram similar with data；
Fig. 2 is the schematic diagram for showing semantic association relationship；
Fig. 3 is the calculation schematic diagram of Hamming distance of the present invention；
Fig. 4 is the flow diagram that Hash codes upload in Hash table of the present invention；
Fig. 5 shows the present invention and is uploaded to storage system to the entire workflow for executing semantic query and returning the result from file Journey；
Fig. 6, which is that the present invention is based on threshold values, carries out the schematic diagram of selection to the side of Hamming figure；
Fig. 7 is the process schematic that the present invention carries out semantic query；
Fig. 8 is the schematic diagram for the separation link Hash table that the present invention uses；
The schematic diagram for the Hamming figure that Fig. 9 present invention constructs.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and It is not used in the restriction present invention.As long as in addition, technical characteristic involved in the various embodiments of the present invention described below Not constituting a conflict with each other can be combined with each other.
In order to solve the problems, such as Content based coding important and urgently to be resolved in storage system, and consider storage system In conventional metadata can not cope with the status of Content based coding, the present invention is proposed the contents semantic information of file itself (Content-based Semantic Information) is fused in the metadata structure of storage system, so that storage System supports the semantic query (Content-based Semantic Query) based on content, referred to as semantic in the present invention It inquires (Semantic Query).The semantic query of image data and file in primary study of the present invention storage system, because Image data is typical unstructured data, and data volume is larger, and semantic information is beyond expression of words, is brought to storage system More acute challenge.The present invention uses picture material Hash as the metadata of storage system semanteme, thus by image text The contents semantic record of part within the storage system, and uses a kind of more efficient Hamming figure (Semantic Hamming Graph) the contents semantic of structure organization image.It is similar that this not only can rapidly calculate the semanteme between image and alternative document Property and query interface can be provided user is facilitated to inquire.Meanwhile it can be quickly found out and return by the index of graph structure It returns as a result, providing more efficient and intelligentized interface and service for storage system.
The target that the present invention works is to realize the semantic query based on content within the storage system by metadata structure.By In metadata very light weight (occupy little space and metadata operation is fast) within the storage system, therefore adds semantic metadata and arrive Also it must assure that lesser expense in storage system, not so can aggravate load of the storage system on calculating, storage and I/O, lead Cause the decline of global storage performance.In view of these factors, the present invention is using the two-value semanteme Hash codes of fixed length as semantic member Data are simultaneously integrated into storage system, wherein semantic Hash codes are obtained by Hash study (learning to hash).Pass through The method of dimensionality reduction and quantization may be implemented to be that a length is fixed (if length is 48 or 96) converting of image file Two-value code.This method greatly reduces the storage overhead of semantic metadata.According further to the characteristic of Hash study, between image Semantic similarity degree can be calculated according to the distance between Hash codes.Since Hash codes are the characteristics of two-value (only comprising 0 With 1), distance can be obtained by Hamming distance operation, to greatly improve the efficiency of semantic query realization.
The metadata system present invention for the content oriented semantic query that the present invention realizes can support semantic coverage to inquire, solution It releases as follows:
Semantic coverage inquiry, which refers to, inquires contents semantic similitude and a specific file (referred to as inquiry dot file) Meet the All Files in a given range.
Method different from the past based on local sensitivity Hash, the present invention are important to notice that Semantic Similarity, such as one The semantic information of image file description, i.e., this iamge description is any content, rather than the data information of shallow-layer (pixel). By taking Fig. 1 as an example, two pairs of images of solid line connection are the image for describing dog and cat respectively, this is semantically similar, that is, is connected Two images are all the same animals (identical content) of description.Two pairs of images of dotted line connection are the similar image of data, this Pixel distribution between two pairs of images is more similar, but the content that they are described is not same species (dog or cat), therefore, this Two images simultaneously do not have Semantic Similarity.
As shown in figure 4, the present invention is based on the image inquiry method of contents semantic metadata the following steps are included:
(1) from image data concentrate obtain image collection, and using learning-oriented hash algorithm to the image collection at Reason, to obtain the Hash label of the image collection；
Specifically, learning-oriented hash algorithm used in this step is depth self study Hash (Deep Self- Taught Hashing, abbreviation DSTH) algorithm, amount of images included by image collection can freely be arranged, and preferably 100 It is a.
Image data set used in this step is CIFAR-10, STL-10 or ImageNet.
(2) using image collection as the input of neural network, the Hash label for the image collection that step (1) is obtained is made For the output of neural network, training is iterated to neural network, to obtain trained neural network；
Specifically, neural network used in this step is SimpleNet neural network.
It is as follows to be iterated trained process, initial parameter (it can be arbitrary value) is set for neural network first, so Image collection is inputted in the neural network afterwards, to be exported as a result, the Hash mark that the output result and step (1) are obtained Label are compared, and the initial parameter of neural network is then updated by back-propagation algorithm, is then iteratively repeated the above process, directly Until the error between output result and Hash label reaches preset threshold.Wherein preset threshold can be freely arranged, if What is be arranged is smaller, then repetitive exercise process obtain result accuracy it is higher, it is on the contrary then lower.
(3) new image collection is obtained from image data set, by image collection input step (2) the trained nerve In network, to obtain the cryptographic Hash of the image collection, the cryptographic Hash of each image in image collection is stored in correspondence image In metadata, and each Hash obtained using separation link Hash table (Separate Chaining Hash Table) expression The mapping relations being worth between corresponding image, as shown in Figure 8；
It can be seen in fig. 8 that the case where corresponding to piece image there are a cryptographic Hash, there is also a cryptographic Hash pair It should be in the multiple image the case where.
(4) all cryptographic Hash in separation link Hash table used according to step (3) construct Hamming figure, as shown in Figure 9；
This step (4) specifically includes following sub-step:
Counter i=1 is arranged in (4-1)；
(4-2) takes out i-th of cryptographic Hash in separation link Hash table, and it is corresponding that i-th cryptographic Hash is established in Hamming figure Node, and the Hamming distance between i-th of cryptographic Hash and each cryptographic Hash having been taken out before this is calculated, from calculating Select minimum value in obtained multiple Hamming distances, using the corresponding all cryptographic Hash of the minimum value as in Hamming figure with this i-th The node that the corresponding node of a cryptographic Hash is connected, between the corresponding each cryptographic Hash of the minimum value and i-th of cryptographic Hash Hamming distance is as the weight for connecting side in Hamming figure between two nodes；
(4-3) judge i whether be in separation link Hash table cryptographic Hash last, if it is process knot Otherwise i=i+1, and return step (4-2) is arranged in beam；
(5) semantic query received from user terminal is requested, and is parsed to semantic query request, to be checked to obtain Image is ask, corresponding cryptographic Hash is obtained in the metadata of the image to be checked；
(6) corresponding node, root are searched in the Hamming figure that step (4) is established according to the cryptographic Hash obtained in step (5) It is investigated that the node and combination Hamming figure that find obtain other all nodes for meeting semantic query request and its corresponding cryptographic Hash, According to separation link Hash table determine the corresponding cryptographic Hash of other all nodes obtained corresponding to image as final inquiry As a result.
Specifically, its for meeting semantic query request is obtained in this step according to the node and combination Hamming figure that find His all nodes and its corresponding cryptographic Hash determine the corresponding Hash of all nodes of other that obtain according to separation link Hash table The corresponding image of value includes following sub-step as final query result:
(6-1) is using the node found as present node；
(6-2) determines the quantity L for all nodes being connected in Hamming figure with present node, and counter j=1 is arranged；
(6-3) judges whether counter j is less than or equal to L, is if it is transferred to step (6-4), is otherwise transferred to step (6- 8)；
(6-4) judges the Hamming distance between j-th of node being connected in Hamming figure with present node and the node found From whether Semantic Similarity threshold value is less than or equal to, if yes then enter step (6-5), else process terminates；
Specifically, Semantic Similarity threshold value can be freely arranged, value is bigger, then the query result finally obtained Accuracy is lower, on the contrary then accuracy is higher, and preferably value is 1 or 2.
J-th of node is put into query result set by (6-5)；
(6-6) judges whether the Hamming distance between j-th of node and the node found is less than Semantic Similarity threshold value, If it is j-th of node is put into node set, subsequently into step (6-7), else process terminates；
Counter j=j+1, and return step (6-3) is arranged in (6-7)；
(6-8) judges whether node set is sky, if it is takes out all nodes in query result set, and according to this Cryptographic Hash inquiry separation link Hash table of a little nodes in Hamming figure, to obtain the corresponding image of all nodes as finally looking into It askes as a result, process terminates；Otherwise it is transferred to step (6-9)；
(6-9) takes out a node as present node at random from node set, by the present node from node set Middle deletion, and return step (6-2).
Learning-oriented Hash is a kind of Hash mapping method based on machine learning, passes through machine learning in mapping process Method keep initial data similitude.Hash codes are suitable for extensive occasion as efficient nearest neighbor search method.Cause What it is for processing is large-scale image file, and compared to data such as texts, the feature of image is more complicated, wants to extract its contents semantic Need bigger calculation amount.Such as text data, bag of words can be used to obtain the contents semantic of text, but scheme As contents semantic is difficult to be obtained with simple or shallow-layer method, the Hash study of picture material semanteme is caused to learn from shallow-layer Mode is gradually converted into deep learning.
Image Hash learning algorithm solves the problems, such as it is how image is encoded to the two-value (only 0 and 1) of regular length. The memory space that the Hash codes of fixed length do not only take up is small, and is highly suitable for the retrieval of large-scale dataset.If two figures As Hash codes approximation, illustrate that the contents semantic of this two iamge descriptions is also more approximate.Hash towards image learns The popular direction of multimedia and computer vision field.The present invention has used depth certainly according to the specific application scenarios of storage system Study Hash (Deep Self-Taught Hashing, abbreviation DSTH) calculates as the core for generating contents semantic Hash codes Method.Reason is as follows:
(1) DSTH has sufficiently used the ability in feature extraction of deep learning, passes through existing network and large-scale dataset On training result, can effectively for image carry out contents semantic inquiry and search.It is quick based on part compared to previous Feel the method for Hash, DSTH can more extract semantic similar information, and the more than similar information of pixel, to be suitable for language Justice inquiry.Experimental section lists DSTH compared to other semantic query accuracy rate comparisons based on LSH method.
(2) realization of DSTH is a kind of mode of self study, i.e., for new image data set, DSTH has not required mark Label.For the scene of storage system magnanimity isomeric data, the tag along sort that system obtains image is extremely difficult.DSTH does not need to deposit The additional information (label) of image data in storage system.Have a supervision hash algorithm compared to rely on tag along sort, DSTH from Mode of learning has more practicability and universality.
(3) DSTH trains and obtains Hash codes using network SimpleNet rapidly and efficiently, the advantage of doing so is that not Only it can also can effectively accelerate the generating process of Hash codes with the efficiency of training for promotion Hash model.For in storage system The scene of transmitting file and metadata, the necessary expense of upload procedure is low, speed is fast, otherwise can the serious property for influencing whole system Energy.The high efficiency of DSTH makes the generation time of Hash codes of short duration, to guarantee the high-performance in metadata upload procedure, also has Conducive to the completion for distributing and storing metadata process.
DSTH is broadly divided into two stages, i.e. Hash label generation phase and hash function learns the stage.
The main task of Hash label generation phase is to provide Hash label for data set, to instruct next stage Sino-Kazakhstan The study of uncommon function.The groundwork in this stage is to be mapped to depth characteristic in Hamming space according to graph structure, is adopted first With the mode of transfer learning by using the network model obtained on pre-training data set (such as ImageNet) (such as GoogLeNet, AlexNet, VGG16) extract data characteristics used.ImageNet is huge as pre-training data set data volume Greatly, many kinds of, feature extraction is carried out using it as model to be guaranteed with stronger accuracy.
After obtaining feature, dimensionality reduction operation is carried out to these features by spectrum hash mode, uses the closest calculation of K first Method constructs graph model, carries out dimensionality reduction by laplacian eigenmaps (Laplacian Eigenmaps, LE) algorithm, then exists Binaryzation is carried out in each dimension using average value as threshold value, obtains 0 or 1 Hash two-value code.By the step, finally Obtained Hash codes remain the Semantic Similarity between original image, and characteristics of image is reduced to the shorter and fixed length of length The bi-level fashion of degree.Since spectrum Hash can be from the distance between global angle metric data, and being reflected this distance In Hash codes, therefore algorithm has stronger generalization ability.
Learn the stage in the hash function of DSTH, will on last stage obtained in two-value Hash codes be used as Hash label, it is logical Learnt after mode end to end, as used herein is that simple convolutional neural networks accelerate training and generate the mistake of Hash codes Journey.After this convolutional neural networks trains, so that it may applied in metadata generation module of the invention.Due to Hash mark Global clustering information is contained in label, therefore the study of hash function has very strong generalization ability.For storage system In scene, the image category used is ever-changing, so can handle multi-source data problem well using DSTH.In addition, whole A process not will use new label, therefore training process can satisfy actual application scenarios.
Because emphasis of the invention is using DSTH rather than proposes and optimize DSTH, the theory and realization of specific DSTH (Liu Y, Song J, Zhou K, et al.Deep self-taught hashing for image can be referred to Retrieval [J] .IEEE transactions on cybernetics, 2018 (99): 1-13), in this bibliography The selection and application process for teaching activation primitive, coding mode and loss function that it is used in detail, in addition also show Advantage of the DSTH compared to other image hash algorithms in accuracy rate.According to document experience, present invention selection uses 48 Code length as semantic metadata.For storage system, 48 two-value Hash codes only account for 6 bytes (48 bit) Length, its storage overhead generated can be ignored for an image.
Although many image search engine (such as Google's picture search with Baidu's picture search) all supports similar image to look into Service is ask, but these search engines are matched highly dependent upon hand labeled image or using writings and image.This people The method of work label means huge expense, and in many scenes, and what this inquiry was found is that data (pixel) are similar Image, rather than semantic similar image.In contrast, DSTH is the deep learning hash method based on extraction of semantics, can By it is manually mark in the case where the automatically mapped image data in the form of two-value Hash codes.
Hamming figure is the data structure for being used to tissue semantic metadata in present system.DSTH can be each image file A Hash codes are generated, so that system can generate a large amount of Hash codes.Semantic query (range query) is according to semantic Hash codes Between relationship return and meet the results of specified conditions.Therefore it for a large amount of Hash codes, needs using efficient organizer Formula constructs data structure and supports inquiry.According to the two-value Hash codes feature that DSTH is generated, the present invention abandons in organizational form Traditional tree structure.Tree structure is as a kind of hierarchical structure, and connection relationship represents catalogue level, but Semantic Similarity Connection represent the semantic association between file.At the same time, the present invention still can retain directory-file layer in traditional directory tree The structure of grade has ten to storage system itself because directory tree structure is inquiry and location structure basic in storage system Divide important meaning.The contents semantic that the present invention enhances heritage storage system simply by semantic metadata tissue is increased is inquired Function.
Substantially, tree construction is a kind of special graph structure, and graph structure is that graph structure can relative to the advantage of tree construction To express the contents semantic similitude between file more flexiblely.For the image file in storage system, file it Between semantic relation be mostly no level, as shown in Fig. 2, file A and B and C are semantic related, file C is related to D semanteme, and File D is semantic related to A, and semantic association relationship can be figured, but this incidence relation is under traditional tree structure Failure.
The advantage of two-value Hash codes is to support efficient semantic association calculating, i.e., the exclusive or fortune under Hamming distance It calculates.The calculating of Hamming distance is as shown in figure 3, the two-value Hash codes 100001010 and other three isometric two-values that length is 9 The manner of comparison of Hash codes 110001010,001001010,10101110 is to compare two different digits of Hash codes, number Hamming distance as between the two Hash codes, such as 100001010 and 110001010 have one it is (therein from from left to right second Position) it is different, so the Hamming distance between them is 1.It is that computer is good at place using the advantage that Hamming distance calculates 01 data are managed, the quick calculated result of exclusive or (XOR) operation can be passed through under less expense.
Under the premise of obtaining All Files Hash codes, the present invention constructs Hamming figure according to the Hamming distance before them (as indicated at 4).For Hamming figure, the node in figure is Hash codes, and the side in figure represents between Hash codes and exists Semantic relation, wherein the weight on side is the Hamming distance between two Hash codes.The size of Hamming distance decides between data Similarity degree, for image, semantic Hash, the semanteme that lesser weight represents two images is even more like or related, instead It, two images then may be dissimilar or uncorrelated.The advantage of Hamming figure be it by composition when expense between floating number The Hamming distance calculated under being changed into 01 calculates, to greatly improve the building speed of figure.
The characteristics of according to Hash codes, different images may be mapped as identical Hash codes, so as to cause in Hamming figure A node correspond to multiple files.In addition, the quantity of corresponding node also will increase when amount of images increases, lead to side Quantity is increased with the number of square rank, so that the storage of Hamming figure needs to consume a large amount of storage resource and computing resource.Needle To two above problem, the metadata pipe of efficient and low overhead is obtained the invention proposes the optimization method of two kinds of Hamming figures Reason method.Both optimization methods are the trimming and selection of the node merging and the Hamming figure side based on threshold value when Hash collides.
Method for organizing of the present invention as metadata, structure are married with storage system.Totality of the invention Frame is as shown in Figure 5.The present invention includes to be uploaded to storage system to the entire work for executing semantic query and returning the result from file Make process.
Due to the present invention towards be image file, it is necessary first to have a large amount of image source (such as image in internet) It uploads files in storage system.The process that it is uploaded is different from conventional store method.The present invention is first by the image text of upload Part is input in trained DSTH network, to obtain the corresponding semantic Hash codes of image.Later, the present invention can be by language Adopted Hash codes and image file upload in storage system correspondingly, and the first number of the semanteme using this Hash codes as image According to.The upload part of storage system image file terminates herein.
After file upload terminates, need to carry out the management of semantic metadata, since different image files may be right Answer the same cryptographic Hash.The present invention is provided with the Hash table of a separation link for the file chained list of identical semantic cryptographic Hash It is chained up, it is used for the file of the identical semantic cryptographic Hash of tissue；As the communication structure of Hamming figure and storage system, this Hash Table (as shown in Figure 5) can be convenient the data information transfer at both ends when executing semantic query.
Hash table judges whether the file semantics Hash codes newly uploaded have occurred in systems, will if not occurring Emerging Hash codes upload in Hamming figure, i.e., Hamming figure does not repeat storage semantic metadata.Hamming figure is semantic member in Fig. 5 The major part of data management, it is realized using chart database at present, the reason is that chart database have high stability and Scalability, and have the dedicated query language of diagram data.The present invention has devised semantic coverage inquiry using these characteristics Interface.
In table 1, the present invention and other metadata systems and file system method can be inquired compare.Of the invention sets There are two aspect is similar with these metadata systems for meter method.On the one hand, they are extracted the feature for inquiry and all with these Feature is metadata；On the other hand, they all employ data structure aggregated metadata to accelerate query process.In metadata During generating and organizing, the method that the present invention uses is different from other metadata systems.
1 present invention of table is compared with other metadata organizations
For these systems, Spyglass and SmartStore are using conventional metadata, and being associated in for metadata is interior It is in appearance and uncorrelated.FAST has used the metadata based on content, but the method for its PCA-SIFT is the shallow-layer spy of image Sign, is not suitable for semantic query.And FAST only can be used in accurate inquiry, may not apply to similarity query.The present invention Depth Hash (DSTH) method used then meets the inquiry of the Semantic Similarity based on content.It is essential in metadata organization It is required that be by " semanteme " similar Content Organizing together, this be actually one cluster process.SmartStore passes through latent The semanteme of metadata is excavated in semantic analysis, is then clustered semantic similar content, but its semantic relation only uses The relationship of simple metadata.Spyglass is then clustered using the method for distinguishing hierarchy, i.e., makes semanteme by tree structure Similar content is as far as possible under a subtree.FAST is to be by the PCA-SIFT Feature Mapping extracted by local sensitivity Hash The cryptographic Hash of LSH, and then can be searched by Bloom filter.Polymerization of the invention is in Hamming space, by Hamming It condenses together apart from similar semantic metadata, so that the requirement of semantic query is more adapted to, therefore compared to tree structure, It has more flexibility and high efficiency.For the organizational form of metadata, FAST solves hash-collision by Cuckoo Hash Problem, and be allowed in conjunction with Bloom filter.SmartStore has used the R tree in space search field by semantic Similar content Building is in the same field of R tree.Spyglass then realizes quick metasearch by K-D tree.Organizational form of the invention For Hamming figure, this mode makes file corresponding with the node in figure, and the semantic complex query for image file may be implemented (range query).
Extraction of semantics module of the present invention is the basis of entire semantic metadata management, because only that being extracted by this module Semantic Hash codes (semantic metadata) can just carry out the management of metadata and the realization of semantic query function later.Language The design of adopted extraction module is broadly divided into three steps: (1) training hash function network (2) is extracted semantic Hash codes (3) and is uploaded With allocated semantics metadata.
In the generation phase of Hash label, DSTH directly use existing deep neural network (such as AlexNet, GoogLeNet, VGG16 etc.) extract data set in characteristics of image.In general, depth network can extract the floating-point of higher-dimension Feature vector.According to the dimension reduction method told about above, these feature binaryzations are obtained into the semantic Hash codes of image.In training During hash function network, according to the Hash codes that DSTH is obtained in Hash label generation phase, simple network is designed SimpleNet is fitted Hash label.Meanwhile according to the analysis of the code length of DSTH as a result, present invention unification selects 48 Hash codes.
During second step semanteme Hash codes are extracted, image file can first pass through before uploading to storage system Trained good neural network.According to the batch processing characteristic of deep learning, the present invention provides two kinds of upload modes: batch uploads It is uploaded with independent.Batch upload refers to is sent to trained SimpleNet for some images in bulk, to obtain simultaneously more The semantic Hash codes of a file.It individually uploads and uploads to SimpleNet with referring to every image sequence, so that neural network is by land Result is exported continuously.
The advantage that batch uploads is that expense is low.On the one hand, deep learning can more efficiently batch execution data；Another party Face, individually upper transmitting file can constantly start network, to cause additional time overhead.Therefore, generally batch uploads meeting It is uploaded better than independent.The shortcomings that batch uploads is to have to just execute until the Hash codes of All Files all generate to finish The work that data allocations and file later upload, brings higher delay to storage system.Storage is tested later The comparison of system upload document time and semantic metadata Mass production time.In the design, the present invention mainly uses on batch The mode of biography.Because of the characteristics of SimpleNet, the speed of Hash codes is generated quickly, batch, which uploads, can guarantee highest performance, And it will not influence the uplink time of storage system.
In the upload and distribution of third step semantic metadata, the present invention will record the upper of semantic Hash codes and image file Biography sequence, to guarantee semantic Hash codes and image file is corresponding.After obtaining the semantic Hash codes of file, the present invention Semantic Hash codes can be accordingly stored in the extended attribute (Extended Attributes) of image file first, herein Need to guarantee that file system supports extended attribute (most file system such as XFS, ext3, NTFS can be supported).Later Swift can automatically turn the semantic Hash codes saved in the extended attribute of file when uploading image file to storage system Become one (referred to as semantic metadata) of metadata attributes.So far, the work of extraction of semantics module terminates.For different Back end storage system, as long as can guarantee that semantic metadata and image file correctly correspond to, so that it may use different metadata Distribution method.
Semantic query of the invention is realized by Hamming figure, and since Hamming figure interior joint is Hash codes, and side represents Hamming distance, therefore in Hamming figure there is no storage file information, do not know which file corresponds to these Hash codes yet. DSTH is a kind of similitude hash algorithm, similar image can be mapped to identical or similar Hash codes.Therefore, a Kazakhstan Uncommon code may be corresponding with multiple images file, needs to manage this incidence relation using a kind of data structure, to realize member The communication of data.
According to Fig.5, it can be found that there are the Hash tables (both ends connection Hash table) of a separation link in the present invention To be connected to semantic metadata management, characteristics of image and storage system.Separation link Hash table is the basic number of metadata communication According to structure, the semantic Hash code value that wherein key assignments of Hash table is existing 48 in storage system, and each key assignments is linked to List structure record and possess this mark of the semantic Hash codes as the image file of metadata.Due in storage system, Its absolute path is unique, therefore the present invention uses absolute path as the mark of file, and list structure is used to carry out chain It connects.Both ends connection Hash table and the relationship of storage system and Hamming figure are illustrated in fig. 6 shown below.
Both ends connect Hash table in system there are no being sky when any image file, and when transmitting file in system, DSTH is raw At image hash code and the absolute path of image file can pass to Hash table simultaneously (see solid arrow in Fig. 6).For each Cryptographic Hash, Hash table use the time of O (1) to search with the presence or absence of the cryptographic Hash in all existing key assignments, if it exists then first Illustrate that the files-designated of this document can be added in the chained list tail portion of its link in the file for having the cryptographic Hash in storage system, Hash table Know.If the result that Hash table is searched is to illustrate not save this Hash codes in storage system there is no this semantic cryptographic Hash File and this file are first, can newly create a key assignments in Hash table in this case, and key assignments content is semantic thus to breathe out Uncommon value, an and then chained list is created on this key assignments, the mark of the first item of chained list file thus.
After chained list insertion finishes, for emerging Hash codes, the present invention can be passed in Hamming figure, thus Hamming figure can newly increase the node of this Hash codes, calculate the relationship with other nodes later, and then add side.The present invention couple Calculating and selection in the side of other nodes are optimized, and optimization process sees below text.It is whole after on Hamming figure side, selection is finished The upload procedure of a file also just terminates.If the Hash codes of the file uploaded have existed in both ends connection Hash table, say The bright figure of Framingham stores this node before upload procedure.In the case, Hamming figure does not do any update.It is advising greatly Modulus is in the case where, and identical situation, which occurs, in cryptographic Hash will increase, and this method can promote the update efficiency of Hamming figure.Make To summarize, the flow chart of upload procedure is illustrated in fig. 4 shown below.
During semantic query, the present invention can obtain the semantic cryptographic Hash of inquiry dot file first, then breathe out this It is uncommon to be sent to Hamming figure.By the inquiry in figure, qualified Hash codes are returned, and then these Hash codes needs are sent to Both ends connect Hash table (see Fig. 6 dotted arrow), and Hash table is according to the available qualified file of the corresponding chained list of Hash codes Name can thus return to the result of semantic query by storage system.For 48 Hash codes of DSTH, Hash codes are most Have 2 more48Kind situation, so Hash collision is not to frequently occur.For key assignments each in Hash table, chained list Length can remain to it is shorter, to reduce the time of inquiry linked list.
Two kinds of optimization methods of the data structure Hamming figure of semantic metadata management in the present invention --- the conjunction of Hash collision And and the Hamming figure side based on threshold value selection.Both methods can make the storage overhead of Hamming figure of the present invention contract significantly Subtract, and can reduce the storage overhead and calculate significantly the time used under large-scale data.
The processing of Hash collision
It is as previously mentioned, DSTH algorithm will appear the case where Hash collides, i.e., not exactly the same image generates complete The same Hash codes.In this case, the present invention only retains the node of a corresponding cryptographic Hash in Hamming figure, and uses Both ends connection Hash table links identical Hash codes file.For DSTH, if the semanteme that different images generate Hash codes are identical, illustrate that two kinds of image, semantics are extremely similar (corresponding Hamming distance is 0).It is illustrated in fig. 8 shown below, the upper half For the image divided by generating identical semantic Hash codes after neural network, the image of lower half portion is also to generate complete phase Same Hash codes.In this case, the image of top half has only generated a Hash codes node, then mitogenetic with lower half At Hash codes node connected with a line, relationship between 8 images has only been meant that out by 2 nodes in this way.
The optimization method merged by Hash codes, can not only pick out most semantic similar image in semantic query, It can be to reduce the storage overhead and computing cost of figure significantly.By Hamming figure by these identical Hash codes file identification chains It picks up and, it is ensured that a Hash codes node is only stored, so that all Hash nodes are all that uniquely, will not repeat to deposit Storage.If Hamming figure is using file as node, then leading to a large amount of redundant node and side, and using figure come according to Hamming distance Search relationship will cause huge expense.Of the invention can guarantee that data will not be redundantly stored using Hash codes as node, from And guarantee the high efficiency of Hamming figure.
The selection on the Hamming figure side based on threshold value
The Hamming figure introduced above has a serious performance bottleneck, i.e., if node and others that Hash codes represent Hash codes node is the state connected entirely, then many sides can be generated.The storage on these sides needs a large amount of expenses, and much While being It is not necessary to stored.If the Hamming distance of two Hash codes is too long, the semantic relevance between them is just It is low, thus they there is no need to be attached with side.This part proposes the selection on the Hamming figure side based on threshold value, so that Hamming Figure only stores the side of most semantic relevance.
If there are N images in storage system, corresponding N number of two-value can be generated by depth Hash model DSTH and breathed out Uncommon code, since there may be multiple identical images and DSTH similar image can be mapped to identical Kazakhstan in storage system Situations such as uncommon code, occurs, and different two-value Hash codes are defined as Nd(≤N, ∈ Z+) a.NdIt is the number of Hamming figure interior joint.By Hamming distance between node is the weight on side between two nodes, the relationship between two nodes be also it is mutual, so Hamming The side of figure is nondirectional.For the Hamming figure G in storage system, it possesses NdA node, the then maximum that it can possess Relationship number isMaximum edge numbers and number of nodes are at quadratic relationship.It illustrates, if only 10,000 different Kazakhstan Uncommon code, then the number on side is up to 49,995,000 between them under full connection, so many side and weight are deposited Storage will bring very big storage and computing cost.
For such situation, the present invention proposes the method based on threshold value to reduce the number on side, for the every of Hamming figure The number that a threshold value T (being exactly minimum value used in above-mentioned steps (4-2)) carrys out binding side is arranged in a node, and N is arrangedd*For Scheme the * node in G, H (Nd*) it is Nd*Semantic cryptographic Hash.(0≤i, j < N so is met for arbitrary integer i, jd), NdiAnd NdjBetween Hamming distance can use hd (Ndi, Ndj) indicate.By XOR operation symbolIt indicates, then can be with With hd (H (Ndi),To calculate Hamming distance.The effect that threshold value T is arranged is NdiWith NdjBetween can connect side and need to meet:
hd(H(Ndi), H (Ndj))≤T
The selection of threshold value T is the tradeoff of information content and efficiency.When the value of T is bigger, then explanation can have more sides can be with Retaining can guarantee that the information of storage is enough so in the figure, also more so as to the inquiry operation of execution.At the same time Bring cost is to will lead to many sides to be stored, and there is no relationship either semantic relation is weak on wherein most side.Such as The value of fruit T is smaller, then can retain the strongest side of Semantic Similarity as far as possible, but will appear the case where losing information. Since semantic query is mainly the file of the strongest Hash codes of query semantics relationship, so the present invention selects to retain T small as far as possible Value.For the scene in this Hamming space, minimum can be with the distance of value for 1, because distance can be merged for 0 cryptographic Hash.It is right For 48 Hash codes, Hamming distance can illustrate that the corresponding file semantics of the two Hash codes are much like for 1, but in reality It has been found that by T value be 1 in testing, then will appear a large amount of isolated node and make Hamming figure that can not carry out semantic query, it is former Because being many nodes other nodes that can not find Hamming distance only be 1 as side connection, so causing the failure of Hamming figure.
Based on this, given threshold T of the present invention changes with different nodes.Possess N for onedThe Hamming figure of a node G sets TiFor the threshold value of i-th of node, then TiValue be NdiIn node and G in the Hamming distance of other all nodes Minimum value ensures that node N in this waydiThe case where at least connecting with a node, not will cause isolated node, Er Qiebao It demonstrate,proves it and is connected to semantic most like node, and can effectively reduce the quantity on side.So for integer i and j,Value can be defined as:
Ti=min { hd (H (Ndi, Ndj)), j ≠ i, j ∈ [0, Nd)
For following figure 6, node 6 is the node for being newly added to Hamming figure, this node needs find the side that can connect, institute The Hamming distance with other all nodes 1,2,3,4,5 can be acquired with it, then acquires the smallest Hamming distance value.In Fig. 6 Smallest hamming distance is 2, so T6Value be 2, it can connect all node (sections i.e. in figure for being 2 with its Hamming distance Point is 1).When the threshold value for determining each node, and even after crack approach, entire semantic metadata upload procedure will be complete At.
The most important function of the present invention is to realize the similar inquiry of semantic relation, and semantic query refers to the semanteme based on content Inquiry, that is, find image similar with specific file on contents semantic.Semantic query of the invention is to be directed to have stored in be This specific file is referred to as to inquire dot file by the inquiry of file in system, the present invention.It is had for each inquiry dot file Corresponding semanteme Hash codes (semantic metadata) can find a specific section according to this semanteme Hash codes in Hamming figure Point, the referred to as query point of semantic query, are shown in Fig. 7.
When the present invention receive semantic query request after, can by request inquiry dot file semantic metadata take out and It is sent to Hamming figure.Hamming figure quickly navigates to the query point in figure, starts to carry out graph traversal and search then to obtain It most like Hash code value and returns the result.In Fig. 7, the Hash codes of inquiry request are sent to the query point in Hamming figure, Then the present invention starts to be layered and inquire.Most like first is file identical with query point Hash codes, then of the invention By the progressive lookup (first layer, the second layer, third layer are successively progressive) of level, the Hash codes of all meet demands are eventually found Value.Take out query result by chained list of these Hash codes in Hash table, wherein the node in Hamming figure be in Hash table Hash key assignments be it is corresponding, the Hash codes of repeated and redundant will not be all stored in two structures.The query search process of Hamming figure It can be understood as a kind of breadth-first search (Breadth-First-Search, abbreviation BFS) of graph structure.Below will The concept and method of semantic coverage inquiry are introduced, and introduces and how inquiry is promoted according to the characteristics of DSTH semanteme hash algorithm Accuracy rate.
Semantic coverage inquiry
Semantic coverage inquiry refers to All Files of the lookup with query point file semantics correlation in a particular range. In the present invention, semantic to be indicated by Hash codes, the correlativity between semanteme is true by the Hamming distance between Hash codes Fixed.The smaller then declarative semantics correlation of Hamming distance is stronger, so semantic coverage inquiry can be defined as looking into the present invention Ask the All Files for being less than or equal to a special value γ with the Hash codes Hamming distance of specific inquiry dot file.This sets Fixed numerical value γ is exactly the query context (Query Range) of semantic coverage inquiry.
It is known in DSTH algorithm, smaller Hamming distance illustrate Hash codes indicate file content semantically get over phase Seemingly, then most like image is identical Hash codes, Hamming distance 0.By largely testing discovery, 48 are being used During Hash codes, when Hamming distance is greater than 2, semantic similarity degree can weaken, and cause the knot for returning to more mistake Fruit.So in an experiment, taking lesser query context (especially in the case where large-scale dataset) that can return more quasi- True query result.
The process of semantic query can be expressed as in Fig. 7, for a semantic coverage inquiry hd≤γ, query process Start from the query point in Hamming figure.The present invention obtains the Hash codes of semantic query dot file first, then in the chain of Hash table All and identical file of query point Hash codes is taken out in table, since semantic coverage inquiry can retrieve Hash codes as 0 institute Have file, by the step for the document result of hd=0 can be returned.
As γ >=1, the process of semantic coverage inquiry not only needs query point, it is also necessary to which Hamming figure is traversed and searched Rope obtains file by the chained list of Hash table to obtain all qualified Hash codes.In this case, query point Hash codes will as figure traversal starting point, then find other results for meeting querying condition using BFS.It obtains first The node (such as Fig. 7) of all BFS first layers is taken then to check whether this node meets the condition of inquiry, qualified Kazakhstan Uncommon code node returns to file by the chained list of Hash table.If needing to be traversed for more numbers of plies in the case where γ >=2, On condition that upper one layer of node, which meets, is less than γ with the Hamming distance of query point, it just will do it traversal, otherwise do not meet traversal Condition.Termination condition is when the level of BFS search is identical as range γ, the reason is that meeting item according to the construction method of Hamming figure The Hash codes of part can be within the node of the same number of plies.
Scalability Analysis
The present invention increases from data volume and two dimensional analysis of point spread scalability of the invention.Increase from data volume From the point of view of, according to experimental result, when data volume increases (from CIFAR-10 data set to ImageNet), semanteme of the invention is looked into It is still stable to ask time overhead, because node of present invention during inquiry near search inquiry point, works as data When amount increases, the present invention can not be limited by data scale, search adjacent node be kept, to there is the stable time Expense.In addition, the Hash table of separation link is also the key that the present invention guarantees scalability in data volume increase, because in number According in increased situation, the repetition of semantic Hash be will increase, so that Hash table can save the memory space of Hamming figure, so that phase It is organized in together with Hash codes, and remains file identification information convenient for inquiry.
It is analyzed from point spread angle, Hamming figure of the invention is currently running on the chart database of single-point, i.e., all Semantic query the reason of all running, be designed in this way on a machine be that Neo4j chart database is only applicable to list at present Machine mode.In view of the availability and fault tolerance of system, when figure can be distributed to multiple machines by present system It waits, the access speed of user can faster, and semantic query service can be more efficient, and when server breaks down, distributed Graph structure service still can be effectively provided, this is also improvement direction of the invention following.The present invention can support to be distributed The realization of formula has used two-server to construct the cluster of OpenStack Swift in an experiment, but due to chart database Characteristic, semantic metadata institutional framework still runs on single server.It can be by being in following work System carries out figure division so that figure distributed storage is on multiple nodes, to realize distributed metadata management, avoids the occurrence of The problem of Single Point of Faliure.
To sum up, the present invention has the advantage that
(1) present invention uses picture material Hash as the metadata of storage system semanteme, thus by image file Hold semantic record within the storage system, and uses a kind of contents semantic of more efficient Hamming graph structure organization chart picture.This It not only can rapidly calculate the Semantic Similarity between image and alternative document and query interface can be provided and facilitate user It is inquired.Meanwhile can be quickly found out and return the result by the index of graph structure, it is provided more efficiently for storage system With intelligentized interface and service；
(2) present invention has used depth self study Hash DSTH come as life according to the specific application scenarios of storage system At the core algorithm of contents semantic Hash codes.DSTH has sufficiently used the ability in feature extraction of deep learning, passes through existing net Training result on network and large-scale dataset can effectively carry out the inquiry and search of contents semantic for image.It compares In the method in the past based on local sensitivity Hash, DSTH can more extract semantic similar information, and more than pixel is similar Information, to be suitable for semantic query.The realization of DSTH is a kind of mode of self study, i.e., for new image data set, DSTH has not required label.For the scene of storage system magnanimity isomeric data, the tag along sort that system obtains image is very tired It is difficult.DSTH does not need the additional information (label) of image data in storage system.Kazakhstan is supervised compared to having for tag along sort is relied on The self study mode of uncommon algorithm, DSTH has more practicability and universality.DSTH is come using network SimpleNet rapidly and efficiently Hash codes are trained and obtain, the advantage of doing so is that not only can also can effectively add with the efficiency of training for promotion Hash model The generating process of fast Hash codes.For the scene of transmitting file in storage system and metadata, upload procedure must expense be low, speed It fastly, otherwise can the serious performance for influencing whole system.The high efficiency of DSTH makes the generation time of Hash codes of short duration, to protect The high-performance in metadata upload procedure is demonstrate,proved, is also beneficial to distribute and store the completion of metadata process.
(3) present invention is using the graph structure of Hamming figure come the data structure of tissue semantic metadata.Traditional directory tree construction All files are managed by tree structure, and semantic similar file is caused to be likely under different catalogues.Use tree-like knot Structure carries out generally requiring to traverse all file directorys to compare Semantic Similarity when semantic query, to bring biggish calculating Expense.For the present invention, the hierarchical relationship of catalogue is not affected, the text only indicated the leaf node of the bottom Part metadata is organized in a manner of scheming again.For Hamming figure, semantic similar node is focused into figure Adjacent region, and then the lookup of similar documents can be carried out by simple map search mode, and do not have to All Files Metadata is traversed, to significantly reduce time overhead needed for inquiry.
(4) optimization method merged in the present invention by Hash codes, can not only pick out most semantic in semantic query Similar image, can be to reduce the storage overhead and computing cost of figure significantly.By Hamming figure by these identical Hash Code file identification is chained up, it is ensured that only store a Hash codes node, thus all Hash nodes be all it is unique, It will not repeat to store.If Hamming figure is using file as node, then leading to a large amount of redundant node and side, and Tu Laigen is used According to Hamming distance search relationship, huge expense will cause.It is of the invention to guarantee that data will not be by using Hash codes as node It repeats to store, to guarantee the high efficiency of Hamming figure.
(5) threshold is arranged for each node of Hamming figure in the number for reducing side the present invention is based on the method for threshold value Value T carrys out the number of binding side, and when the value of T is bigger, then explanation can have more sides that can retain can guarantee so in the figure The information of storage is enough, also more so as to the inquiry operation of execution.Bring cost is to will lead to much at the same time While there is no relationship either semantic relation is weak on wherein most side by storing.If the value of T is smaller, can use up The case where strongest side of Semantic Similarity may be retained, but will appear loss information.Since semantic query is mainly inquired The file of the strongest Hash codes of semantic relation, so the present invention selects to retain T value small as far as possible.So as to greatly reduce side and Very big storage and computing cost brought by the storage of weight.
As it will be easily appreciated by one skilled in the art that the foregoing is merely illustrative of the preferred embodiments of the present invention, not to The limitation present invention, any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should all include Within protection scope of the present invention.

Claims (7)

1. a kind of image inquiry method based on contents semantic metadata, which comprises the following steps:
(1) it is concentrated from image data and obtains image collection, and the image collection is handled using learning-oriented hash algorithm, with Obtain the Hash label of the image collection；
(2) using image collection as the input of neural network, the Hash label for the image collection that step (1) is obtained is as mind Output through network is iterated training to neural network, to obtain trained neural network；
(3) new image collection is obtained from image data set, by image collection input step (2) the trained neural network In, to obtain the cryptographic Hash of the image collection, the cryptographic Hash of each image in image collection is stored in first number of correspondence image In, and use the mapping relations between the corresponding image of each cryptographic Hash that separation link Hash table indicates；
(4) all cryptographic Hash in separation link Hash table used according to step (3) construct Hamming figure；
(5) semantic query received from user terminal is requested, and is parsed to semantic query request, to obtain figure to be checked Picture obtains corresponding cryptographic Hash in the metadata of the image to be checked.
(6) corresponding node is searched in the Hamming figure that step (4) is established according to the cryptographic Hash obtained in step (5), according to looking into The node and combination Hamming figure found obtains other all nodes for meeting semantic query request and its corresponding cryptographic Hash, according to Separation link Hash table determines image corresponding to the corresponding cryptographic Hash of other all nodes obtained as final query result.
2. image inquiry method according to claim 1, which is characterized in that learning-oriented hash algorithm is that depth self study is breathed out Uncommon algorithm, image data set is CIFAR-10, STL-10 or ImageNet.
3. image inquiry method according to claim 1 or 2, which is characterized in that neural network used in step (2) is SimpleNet neural network.
4. image inquiry method as claimed in any of claims 1 to 3, which is characterized in that execute repetitive exercise Process is as follows, and initial parameter is arranged for neural network first, then inputs image collection in the neural network, to be exported As a result, the output result is compared with the Hash label that step (1) obtains, nerve is then updated by back-propagation algorithm The initial parameter of network, is then iteratively repeated the above process, until the error between output result and Hash label reaches default Until threshold value.
5. image inquiry method according to claim 1, which is characterized in that step (4) specifically includes following sub-step:
Counter i=1 is arranged in (4-1)；
(4-2) takes out i-th of cryptographic Hash in separation link Hash table, and the corresponding section of i-th of cryptographic Hash is established in Hamming figure Point, and calculate the Hamming distance between i-th of cryptographic Hash and each cryptographic Hash having been taken out before this, from being calculated Multiple Hamming distances in select minimum value, using the corresponding all cryptographic Hash of the minimum value as in Hamming figure with this i-th breathe out The node that the uncommon corresponding node of value is connected, the Hamming between the corresponding each cryptographic Hash of the minimum value and i-th of cryptographic Hash Distance is as the weight for connecting side in Hamming figure between two nodes；
(4-3) judge i whether be separation link Hash table in cryptographic Hash last, if it is process terminates, no I=i+1, and return step (4-2) are then set.
6. image inquiry method according to claim 1, which is characterized in that in step (6) simultaneously according to the node found Other all nodes for meeting semantic query request and its corresponding cryptographic Hash are obtained in conjunction with Hamming figure, and Hash is linked according to separation It includes following sub-step that table, which determines image corresponding to the corresponding cryptographic Hash of other all nodes obtained as final query result, It is rapid:
(6-1) is using the node found as present node；
(6-2) determines the quantity L for all nodes being connected in Hamming figure with present node, and counter j=1 is arranged；
(6-3) judges whether counter j is less than or equal to L, is if it is transferred to step (6-4), is otherwise transferred to step (6-8)；
(6-4) judges that the Hamming distance between j-th of node being connected in Hamming figure with present node and the node found is No to be less than or equal to Semantic Similarity threshold value, if yes then enter step (6-5), else process terminates；
J-th of node is put into query result set by (6-5)；
(6-6) judges whether the Hamming distance between j-th of node and the node found is less than Semantic Similarity threshold value, if It is that j-th of node is put into node set, subsequently into step (6-7), else process terminates；
Counter j=j+1, and return step (6-3) is arranged in (6-7)；
(6-8) judges whether node set is sky, if it is takes out all nodes in query result set, and according to these sections Cryptographic Hash inquiry separation link Hash table of the point in Hamming figure, to obtain the corresponding image of all nodes as final inquiry knot Fruit, process terminate；Otherwise it is transferred to step (6-9)；
(6-9) takes out a node as present node at random from node set, which is deleted from node set It removes, and return step (6-2).
7. a kind of image query systems based on contents semantic metadata characterized by comprising
First module obtains image collection for concentrating from image data, and using learning-oriented hash algorithm to the image collection It is handled, to obtain the Hash label of the image collection；
Second module, for using image collection as the input of neural network, the Kazakhstan for the image collection that the first module is obtained Uncommon output of the label as neural network, is iterated training to neural network, to obtain trained neural network；
The image collection is inputted the second module and trained by third module for obtaining new image collection from image data set Neural network in, to obtain the cryptographic Hash of the image collection, the cryptographic Hash of each image in image collection is stored in correspondence In the metadata of image, and use reflecting between the corresponding image of each cryptographic Hash that separation link Hash table indicates Penetrate relationship；
4th module, all cryptographic Hash in separation link Hash table for being used according to third module construct Hamming figure；
5th module parses semantic query request, for receiving the semantic query request from user terminal to obtain Image to be checked is taken, corresponding cryptographic Hash is obtained in the metadata of the image to be checked；
6th module, it is corresponding for being searched in the Hamming figure that the 4th module is established according to the cryptographic Hash obtained in the 5th module Node according to the node that finds and combines Hamming figure to obtain to meet other all nodes of semantic query request and its corresponding Cryptographic Hash, according to separation link Hash table determine the corresponding cryptographic Hash of other all nodes obtained corresponding to image as most Whole query result.
CN201910546661.7A 2019-06-24 2019-06-24 Image query method and system based on content semantic metadata Active CN110413807B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910546661.7A CN110413807B (en) 2019-06-24 2019-06-24 Image query method and system based on content semantic metadata

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910546661.7A CN110413807B (en) 2019-06-24 2019-06-24 Image query method and system based on content semantic metadata

Publications (2)

Publication Number Publication Date
CN110413807A true CN110413807A (en) 2019-11-05
CN110413807B CN110413807B (en) 2021-04-20

Family

ID=68359603

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910546661.7A Active CN110413807B (en) 2019-06-24 2019-06-24 Image query method and system based on content semantic metadata

Country Status (1)

Country Link
CN (1) CN110413807B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111078914A (en) * 2019-12-18 2020-04-28 书行科技(北京)有限公司 Method and device for detecting repeated pictures
CN112148680A (en) * 2020-09-27 2020-12-29 极道科技（北京）有限公司 File system metadata management method based on distributed graph database
CN112395440A (en) * 2020-11-24 2021-02-23 华中科技大学 Caching method, efficient image semantic retrieval method and system

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6757686B1 (en) * 2000-06-14 2004-06-29 International Business Machines Corporation Method and apparatus for representing database and query information using interval hash tree
US20090324026A1 (en) * 2008-06-27 2009-12-31 Palo Alto Research Center Incorporated System and method for finding a picture image in an image collection using localized two-dimensional visual fingerprints
CN101710334A (en) * 2009-12-04 2010-05-19 大连理工大学 Large-scale image library retrieving method based on image Hash
CN101894130A (en) * 2010-06-08 2010-11-24 浙江大学 Sparse dimension reduction-based spectral hash indexing method
CN104199922A (en) * 2014-09-01 2014-12-10 中国科学院自动化研究所 Large-scale image library retrieval method based on local similarity hash algorithm
CN106777388A (en) * 2017-02-20 2017-05-31 华南理工大学 A kind of multilist hashing image search method of dual compensation

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6757686B1 (en) * 2000-06-14 2004-06-29 International Business Machines Corporation Method and apparatus for representing database and query information using interval hash tree
US20090324026A1 (en) * 2008-06-27 2009-12-31 Palo Alto Research Center Incorporated System and method for finding a picture image in an image collection using localized two-dimensional visual fingerprints
CN101710334A (en) * 2009-12-04 2010-05-19 大连理工大学 Large-scale image library retrieving method based on image Hash
CN101894130A (en) * 2010-06-08 2010-11-24 浙江大学 Sparse dimension reduction-based spectral hash indexing method
CN104199922A (en) * 2014-09-01 2014-12-10 中国科学院自动化研究所 Large-scale image library retrieval method based on local similarity hash algorithm
CN106777388A (en) * 2017-02-20 2017-05-31 华南理工大学 A kind of multilist hashing image search method of dual compensation

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
HUA Y等: "smartstore:a new metadata organization paradigm with semantic-awareness for next-generation file systems", 《IN：PROCEEDINGS OF CONFERENCE ON HIGH PERFERENCE COMPUTING NETWORKING,STORAGE AND ANANLYSIS》 *
LEUNG W等: "Spyglass:fast,scalable metadata search for large-scale storage systems", 《CA：USENIX》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111078914A (en) * 2019-12-18 2020-04-28 书行科技(北京)有限公司 Method and device for detecting repeated pictures
CN111078914B (en) * 2019-12-18 2023-04-18 书行科技(北京)有限公司 Method and device for detecting repeated pictures
CN112148680A (en) * 2020-09-27 2020-12-29 极道科技（北京）有限公司 File system metadata management method based on distributed graph database
CN112148680B (en) * 2020-09-27 2024-04-05 极道科技（北京）有限公司 File system metadata management method based on distributed graph database
CN112395440A (en) * 2020-11-24 2021-02-23 华中科技大学 Caching method, efficient image semantic retrieval method and system

Also Published As

Publication number Publication date
CN110413807B (en) 2021-04-20

Similar Documents

Publication Publication Date Title
US10013438B2 (en) Distributed image search
US20210342350A1 (en) System and method for reducing resource usage in a data retrieval process
CN103838833A (en) Full-text retrieval system based on semantic analysis of relevant words
CN111125422A (en) Image classification method and device, electronic equipment and storage medium
CN108509543B (en) Streaming RDF data multi-keyword parallel search method based on Spark Streaming
CN114218400A (en) Semantic-based data lake query system and method
CN110390352A (en) A kind of dark data value appraisal procedure of image based on similitude Hash
CN110413807A (en) A kind of image inquiry method and system based on contents semantic metadata
Lee et al. MAP-based image tag recommendation using a visual folksonomy
CN113190687B (en) Knowledge graph determining method and device, computer equipment and storage medium
CN113961528A (en) Knowledge graph-based file semantic association storage system and method
Roopak et al. OntoKnowNHS: ontology driven knowledge centric novel hybridised semantic scheme for image recommendation using knowledge graph
EP4163805A1 (en) Graph-based labeling of heterogenous digital content items
CN114218472A (en) Intelligent search system based on knowledge graph
US10628421B2 (en) Managing a single database management system
CN105183845A (en) ERVQ image indexing and retrieval method in combination with semantic features
Duan et al. Distributed in-memory vocabulary tree for real-time retrieval of big data images
US11914641B2 (en) Text to color palette generator
KR101955376B1 (en) Processing method for a relational query in distributed stream processing engine based on shared-nothing architecture, recording medium and device for performing the method
Qi et al. Clustering remote RDF data using SPARQL update queries
CN116523041A (en) Knowledge graph construction method, retrieval method and system for equipment field and electronic equipment
CN114428834B (en) Retrieval method, retrieval device, electronic equipment and storage medium
CN116361428A (en) Question-answer recall method, device and storage medium
US11687514B2 (en) Multimodal table encoding for information retrieval systems
Bouhlel et al. Semantic-aware framework for mobile image search

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant