CN106156319A - Telescopic distributed resource description framework data storage method and device - Google Patents
Telescopic distributed resource description framework data storage method and device Download PDFInfo
- Publication number
- CN106156319A CN106156319A CN201610523135.5A CN201610523135A CN106156319A CN 106156319 A CN106156319 A CN 106156319A CN 201610523135 A CN201610523135 A CN 201610523135A CN 106156319 A CN106156319 A CN 106156319A
- Authority
- CN
- China
- Prior art keywords
- memory node
- data
- memory
- predicate
- subject
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
- G06F16/316—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/38—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the present invention provides a kind of telescopic distributed resource description framework data storage method and device, and described method includes: be assigned in the internal memory of at least two memory node store by all resource description framework RDF data；Data in the internal memory of each first memory node are backuped in the persistent storage device of the second memory node, described first memory node and described second memory node are any one memory node in described at least two memory node, and described second memory node is adjacent with described first memory node.Use the telescopic distributed resource description framework data storage method and device that the embodiment of the present invention provides, the efficiency of data storage and inquiry can be promoted, and ensure when certain memory node lost efficacy, can be rapidly from the adjacent storage nodes local recovery data of this memory node, it is ensured that data storage and inquiry service are persistently provided.
Description
Technical field
The present invention relates to computer realm, particularly relate to a kind of telescopic distributed resource description framework data storage
Method and device.
Background technology
Semantic net is to add, by the document on WWW, the semanteme (Meta Data) that can be understood by machine, so that
Whole the Internet becomes a general information medium of exchange.Resource description framework (Resource Description
Framework, be called for short: RDF) be for describe the World Wide Web Consortium of Internet resources (World Wide Web Consortium,
It is called for short: W3C) standard, for expressing the metadata about internet web page resource, the such as title of webpage, author, amendment day
Phase, content and copyright information etc..Based on RDF data, semantic net is studied, always the study hotspot of scientific circles.And such as
What high-performance, the easily storage of extension ground and inquiry magnanimity RDF data become the problem needing solution badly.
In prior art, for the RDF data of magnanimity, academia has been developed for the RDF data storage side of some maturations
Method, these storage methods mostly use traditional centralized relevant database to store RDF data.But, along with RDF data
Growth rapidly, traditional centralized relational data storage mode cannot meet storage and inquiry calculate demand, because of
This, the storage of distributed RDF data has obtained increasing application with inquiry system.Such as, a kind of RDF data storage scheme
In, use key assignments (key-value) to store the system Redis storage system as RDF data, by passing through many by RDF data
Individual memory node stores, parallel processing query task, thus improves search efficiency.
But, this distributed RDF data storage method provided in prior art, when some data memory node loses
During effect, data will be lost, and causes service disruption.
Summary of the invention
The embodiment of the present invention provides a kind of telescopic distributed resource description framework data storage method and device, uses
In solving the problem that in resource description framework data storage method of the prior art, memory node inefficacy causes service disruption.
The embodiment of the present invention provides the peripheral hardware management and control detection method of a kind of intelligent terminal, including:
It is assigned in the internal memory of at least two memory node store by all resource description framework RDF data；
Data in the internal memory of each first memory node are backuped in the persistent storage device of the second memory node, institute
State the first memory node and described second memory node is any one memory node in described at least two memory node, institute
State the second memory node adjacent with described first memory node.
On the basis of above-described embodiment, described method also includes:
Monitor the storage state of described at least two memory node；
If the storage state of described first memory node was for losing efficacy, then, by the persistent storage of described second memory node
In device, the data in the internal memory of described first memory node of backup, return in the internal memory of described second memory node.
On the basis of above-described embodiment, the described at least two that is assigned to by all resource description framework RDF data stores
Before the internal memory of node stores, also include:
Obtain the subject of described RDF data, predicate and object；
According to described subject, predicate and object, described RDF data is set up index；
Described being assigned in the internal memory of at least two memory node by all RDF data stores, including:
According to concordance hash algorithm, the described RDF data establishing index is assigned to described at least two memory node
Internal memory in store.
On the basis of above-described embodiment, described according to described subject, predicate and object, described RDF data is set up rope
Draw, including the combination of following any one or more mode:
With subject and predicate as key, described RDF data is set up index for value by object；
With subject and object as key, described RDF data is set up index for value by predicate；
With predicate and object as key, described RDF data is set up index for value by subject；
With subject as key, described RDF data is set up index for value by predicate and object；
With predicate as key, described RDF data is set up index for value by subject and predicate；
With object as key, described RDF data is set up index for value by subject and predicate.
The embodiment of the present invention also provides for a kind of telescopic distributed resource description framework data storage device, including:
Distribution module, for being assigned to all resource description framework RDF data in the internal memory of at least two memory node
Store；Described at least two memory node includes the first memory node and the second memory node；
Backup module, for backuping to described second memory node by the data in the internal memory of described first memory node
In persistent storage device, described second memory node is adjacent with described first memory node.
On the basis of above-described embodiment, described device also includes:
Monitoring module, for monitoring the storage state of described at least two memory node；
Data recovery module, for monitoring the storage state of described first memory node for losing efficacy when described monitoring module
Time, by the data in the internal memory of described first memory node of backup in the persistent storage device of described second memory node, extensive
Answer in the internal memory of described second memory node.
On the basis of above-described embodiment, described device also includes:
Acquisition module, deposits for all resource description framework RDF data being assigned at least two in described distribution module
Before the internal memory of storage node stores, obtain the subject of described RDF data, predicate and object；
Memory storage management module, for setting up index according to described subject, predicate and object to described RDF data；
Described distribution module, for being assigned to institute according to concordance hash algorithm by the described RDF data establishing index
State in the internal memory of at least two memory node and store.
On the basis of above-described embodiment, described memory storage management module, specifically for:
With subject and predicate as key, described RDF data is set up index for value by object；
With subject and object as key, described RDF data is set up index for value by predicate；
With predicate and object as key, described RDF data is set up index for value by subject；
With subject as key, described RDF data is set up index for value by predicate and object；
With predicate as key, described RDF data is set up index for value by subject and predicate；
With object as key, described RDF data is set up index for value by subject and predicate.
The telescopic distributed resource description framework data storage method and device that the embodiment of the present invention provides, passes through
All RDF data are assigned to each memory node, improve data storage and the efficiency accessed.Meanwhile, memory node is supported
Be dynamically added and exit, and the data in each memory node are backuped to the adjacent storage nodes in concordance Hash ring
In, it is ensured that when certain memory node lost efficacy, it is possible to rapidly from the adjacent storage nodes local recovery data of this memory node, protect
Card system is not collapsed, and persistently provides data storage and inquiry service.When newly-increased memory node, data are assisted according to concordance Hash
View Autonomic Migration Framework, newly-increased node can come into operation rapidly.Meanwhile, use Redis cluster as the memory storage system of bottom,
And use SP_O, SO_P, OP_S, S_OP, P_SO, O_SP 6 index, and construct distributed RDF query on this basis
Engine, it is achieved that the distributed RDF system of a quick search, and any kind of data query can be supported.
Accompanying drawing explanation
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
The accompanying drawing used required in having technology to describe does one and introduces simply, it should be apparent that, the accompanying drawing in describing below is this
Some bright embodiments, for those of ordinary skill in the art, on the premise of not paying creative work, it is also possible to
Other accompanying drawing is obtained according to these accompanying drawings.
Fig. 1 is the storage of the embodiment of the present invention telescopic distributed resource description framework data and the structure of inquiry system
Schematic diagram；
Fig. 2 is the structural representation of the query plan tree that inquiry plan generation module generates in the embodiment of the present invention；
Fig. 3 is the flow process signal of the embodiment of the present invention telescopic distributed resource description framework data storage method
Figure；
Fig. 4 is that the embodiment of the present invention telescopic distributed resource description framework data storage method carries out data distribution
And the schematic flow sheet of backup；
Fig. 5 is that the embodiment of the present invention telescopic distributed resource description framework data storage method is in memory node
The process schematic of distribution data；
Fig. 6 is the structural representation of the embodiment of the present invention telescopic distributed resource description framework data storage device
Figure.
Detailed description of the invention
For making the purpose of the embodiment of the present invention, technical scheme and advantage clearer, below in conjunction with the embodiment of the present invention
In accompanying drawing, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is
The a part of embodiment of the present invention rather than whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art
The every other embodiment obtained under not making creative work premise, broadly falls into the scope of protection of the invention.
The telescopic distributed resource description framework data storage method and device of embodiment of the present invention intelligent terminal,
For realizing resource description framework (Resource Description Framework is called for short RDF) based on key assignments point of data
Cloth storage and inquiry, improve data reading performance using redundancy, simultaneously by backing up data in adjacent storage nodes so that certain is deposited
During storage node failure, data can be recovered from adjacent storage nodes, prevent system crash, service disruption, affect data storage
With query performance.
Fig. 1 is the storage of the embodiment of the present invention telescopic distributed resource description framework data and the structure of inquiry system
Schematic diagram.
Referring to Fig. 1, the telescopic distributed resource description framework data that the embodiment of the present invention provides stores and looks into
Inquiry system includes main controlled node 10, and at least two is from node 20, and is used for realizing described main controlled node 10 with described from node 20
Between communication communication module 30.Described main controlled node 10 and can be that there is data-handling capacity and storage capacity from node 20
Equipment, such as computer.Described main controlled node 10 includes: RDF data resolver 11, dictionary 12, RDF data statistical module 13,
RDF data sheer 14, query statement resolver 15, inquiry plan generation module 16 and Optimizing Queries module 17.Described from joint
Point 20 includes local search processor 21, memory storage management module 22, backs up and recover module 23, disk storage management module
24 and monitoring module 25.
Specifically, described RDF data resolver 11 is responsible for extracting subject, meaning from the RDF data file of different-format
Language and object (subject, predicate, object) tlv triple.RDF data source is generally from the forms such as NT, N3, TTL at present
Data file.They describe RDF data figure in a different manner, but their basic element is all tlv triple.Described RDF
Data parser 11 is capable of identify that the data file of these forms, and extracts tlv triple from every statement.One data literary composition
The big then 10G of part, described RDF data resolver 11 can rapidly process these files.
Described dictionary 12 is for being mapped to ID by the tlv triple of string format, in order to encodes tlv triple and arranges
Sequence, and save memory space.First, from described RDF data resolver 11, the tlv triple of string format, then root are received
According to each tuple to dictionary is searched corresponding mark (Ientification is called for short ID).Should if there is the most directly returning
ID, if there is no then generating a unique ID, and is stored in corresponding with described character string for described ID in dictionary, returns described
ID。
Described RDF data type statistics module 13 is used for adding up the quantity of the tlv triple of various combination, and by statistical data
It is saved in the internal memory of described main controlled node 10.Described statistical data provides help for the optimization of future query plan, with optimum
Change the order of connection, reduce the generation of intermediate data.In view of subject (be called for short S), predicate (is called for short P), and object (is called for short O), subject and
Predicate (is called for short SP), and the tlv triple of predicate and object (being called for short PO) type is the most conventional, and described data type is added up
Module 13 stores identical S, identical P, identical O, identical SP or the quantity of identical PO, needs to determine that connection is suitable in query script
During sequence, judge according to these statistical datas.The first connection that quantity is few, the rear connection that quantity is many, thus improve search efficiency.
Described RDF data sheer 14 is for being assigned to each according to concordance Hash rule from node 20 by data.
Owing to concordance hash algorithm supports being dynamically added and exiting of node, therefore system can be realized by concordance hash algorithm
Fault tolerance.First build an annular hash space, then RDF data is hashing onto in ring, then memory node is breathed out
Wish in ring, according to clockwise, data are stored in adjacent memory node.When deleting a memory node, according to
Clockwise, will be routed in next memory node by the data of abreviation point.When a newly-increased memory node, will be newly-increased
Data between node and the most adjacent memory node thereof are all routed in newly-increased memory node.
Described query statement resolver 15 is for by query language and data acquisition protocols (Simple Protocol and
RDF Query Language, is called for short: SPARQL) query statement resolves to a query graph, and identifies variable therein, often
Amount and query pattern.
Described inquiry plan generation module 16 is for generating an inquiry plan according to query statement, and inquiry plan is one
Operation tree, on tree, each node is an operator, and operator includes scan operation, attended operation.
Fig. 2 is the structural representation of the query plan tree that inquiry plan generation module generates in the embodiment of the present invention.Such as Fig. 2
Shown in, DIS represents scan operation, finds, from storage, the query pattern met, and DMJ represents and is attached two query patterns
Operation, all of result is attached by last DHJ, generates final inquiry plan.
Described Optimizing Queries module 17, for according to the described inquiry plan generated and described statistical data, optimizes scanning
And the order of connection, enable the operation executed in parallel on query plan tree, and reduce the generation of intermediate object program as far as possible.
The described described local search processor 21 from node 20 receives from described main controlled node 10 for basis
Described inquiry plan performs relevant scan operation and attended operation, and responsible each mutually transmits middle junction between node 20
Really.Wherein scan operation represent according to described query pattern from storage find qualified tlv triple, attended operation represents
Two query patterns are calculated cartesian product and obtains intermediate object program or final result.
The RDF tlv triple data that described memory storage management module 22 is used for receiving are with key assignments (key-value)
Mode is stored in internal memory, and searches data rapidly according to search request.Currently for the possible situation of inquiry, set up altogether 6
Class indexes.Respectively: SP is key, O is value；SO is key, and P is value；OP is key, and S is value；S is key, and OP is value；P is key, and SO is
Value；O is key, and SP is value.Data after index are stored in Redis data base with (String, Set<String>) character string structure
In.
Described backup and recovery module 23 are for storing described according to concordance hash algorithm from the internal memory of node 20
Data backup to the adjacent persistent storage device from node, such as in disk.Data in disk with key value structure by disk
Management module 24 is responsible for organization and management.Until all data have loaded, backup completes the most accordingly.Lose from node 20 when described
Data can be recovered from the disk of node from adjacent rapidly, it is ensured that inquiry service is not interrupted during effect.
Described monitoring module 25 be used for monitoring each from node 20, once any one lost efficacy from node 20, fast notification
Described main controlled node 10 recovers described inefficacy data from node.After described main controlled node 10 receives request, notify that other are deposited
That lives recovers data from nodal parallel, it is achieved fault-tolerant.Specifically, described monitoring module 24 carries out failure detection and make use of
The configuration management of Zookeeper and group membership management service.
Fig. 3 is the flow process signal of the embodiment of the present invention telescopic distributed resource description framework data storage method
Figure.
Refer to Fig. 3, the telescopic distributed resource description framework data storage method that the embodiment of the present invention provides
Including:
S101: be assigned in the internal memory of at least two memory node store by all resource description framework RDF data；
Described memory node is in the storage of above-mentioned telescopic distributed resource description framework data and inquiry system
From node.Specifically, described memory node can be the electronic equipment with storage function, such as computer, mobile terminal etc..
Before performing described step S101, after main controlled node receives RDF data, first obtain described RDF data
Subject, predicate and object；Then according to described subject, predicate and object, described RDF data is set up index.
Described being assigned in the internal memory of at least two memory node by all RDF data stores, including:
According to concordance hash algorithm, the described RDF data establishing index is assigned to described at least two memory node
Internal memory in store.
Specifically, described according to described subject, predicate and object to described RDF data set up index, including following arbitrarily
The combination of one or more modes:
With subject and predicate as key, described RDF data is set up index for value by object；
With subject and object as key, described RDF data is set up index for value by predicate；
With predicate and object as key, described RDF data is set up index for value by subject；
With subject as key, described RDF data is set up index for value by predicate and object；
With predicate as key, described RDF data is set up index for value by subject and predicate；
With object as key, described RDF data is set up index for value by subject and predicate.
Use above-mentioned six kinds of modes to set up index, cover RDF data subject, predicate, object tlv triple according to key assignments knot
The combination in any mode of structure so that the distributed resource description framework data storage method that the embodiment of the present invention provides can be propped up
Hold any kind of data query.
S102: the data in the internal memory of each first memory node are backuped to the persistent storage device of the second memory node
In, described first memory node and described second memory node are any one the storage joint in described at least two memory node
Point, described second memory node is adjacent with described first memory node.
Further, described method also includes:
Monitor the storage state of described at least two memory node；
If the storage state of described first memory node was for losing efficacy, then, by the persistent storage of described second memory node
In device, the data in the internal memory of described first memory node of backup, return in the internal memory of described second memory node.
Fig. 4 is that the embodiment of the present invention telescopic distributed resource description framework data storage method carries out data distribution
And the schematic flow sheet of backup.
Referring to Fig. 3, when carrying out data distribution, main controlled node parses subject, predicate and object from RDF data file
Tlv triple, sets up the index of above-mentioned 6 kinds of key assignmentses to every tlv triple, and according to concordance Hash agreement by RDF data
It is distributed in multiple corresponding memory node preserve.After multiple memory nodes receive data, storage is a to internal memory, and according to
Concordance Hash agreement finds this memory node adjacent storage nodes on concordance Hash ring, backs up data to this adjacent
The persistent storage device of memory node, such as in disk.
Fig. 5 is that the embodiment of the present invention telescopic distributed resource description framework data storage method is in memory node
The process schematic of distribution data.
As a example by Fig. 5, distribution, backup and the recovery process of data is illustrated below.
Refer to Fig. 5, when data are allocated, first, build an annular hash space, then by all RDF numbers
According to being hashing onto in ring, then each memory node is hashing onto in ring, according to clockwise, data is stored to adjacent storage
In node.Such as Fig. 3, illustratively, data 1-3 are stored in the internal memory of memory node A, and data 4-6 are stored in the interior of memory node B
In depositing, data 7-9 are stored in the internal memory of memory node C, and data 10-12 are stored in the internal memory of memory node D, class successively
Push away.Meanwhile, data 1-3 back up in the disk of adjacent storage nodes B of memory node A, and data 4-6 back up in memory node B
Adjacent storage nodes C disk in, data 7-9 back up in the disk of memory node D, data 10-12 back up in storage joint
In the disk of some A, the like.
When memory node D lost efficacy, and needed to delete memory node D, system is in the following manner to the number in memory node D
According to recovering:
1) from the disk of memory node A, data 10-12 are recovered in internal memory；
2) all data 10-12 in memory node A disk and 1-3 are transferred in the disk of memory node B；
3) all data in the disk of memory node A are emptied；
4) all data in memory node C internal memory are backuped in the disk of memory node A.
Illustratively, please continue to refer to Fig. 4, when distributed system PC cluster ability and storage capacity are not enough, need newly-increased
During memory node D, the data migration process in each memory node is as follows:
1) calculate the cryptographic Hash of newly-increased memory node D, determine that memory node is at Hash ring according to the cryptographic Hash of memory node D
In position；
2) from the disk of memory node B, the data in the memory range of memory node D are returned in memory node D
In depositing；
3) data in the memory range of memory node C will be transferred to by the data of the disk of memory node A storage joint
In the disk of some D；
4) data in the disk of situation memory node A；
5) from the disk of memory node B, the data in the memory range of memory node D are transferred to the magnetic of memory node A
In dish；
6) from the internal memory of memory node A, delete the data in the memory range of memory node D；
7) from the disk of memory node B, delete the data in the memory range of memory node D.
The telescopic distributed resource description framework data storage method that the embodiment of the present invention provides, passes through concordance
All RDF data are assigned to each memory node by hash algorithm, improve the efficiency that data storage accesses.Meanwhile, support to deposit
Store up being dynamically added and exiting of node, and the data in each memory node are backuped to the adjacent storage in concordance Hash ring
In node, it is ensured that when certain memory node lost efficacy, it is possible to rapidly from the adjacent storage nodes local recovery number of this memory node
According to, it is ensured that system is not collapsed, and persistently provides data storage and inquiry service.When newly-increased memory node, data are according to concordance
Hash agreement Autonomic Migration Framework, newly-increased node can come into operation rapidly.Meanwhile, employing Redis cluster is deposited as the internal memory of bottom
Storage system, and use SP_O, SO_P, OP_S, S_OP, P_SO, O_SP 6 index, and construct distributed on this basis
RDF query engine, it is achieved that the distributed RDF system of a quick search, and any kind of data query can be supported.
Fig. 6 is the structural representation of the embodiment of the present invention telescopic distributed resource description framework data storage device
Figure.
Refer to Fig. 6, the telescopic distributed resource description framework data storage device that the embodiment of the present invention provides,
At least include distributing module 610 and backup module 620.
Described distribution module 610, for being assigned at least two memory node by all resource description framework RDF data
Internal memory stores；Described at least two memory node includes the first memory node and the second memory node；
Described backup module 620, deposits for the data in the internal memory of described first memory node are backuped to described second
In the persistent storage device of storage node, described second memory node is adjacent with described first memory node.
On the basis of above-described embodiment, further, described device also includes:
Monitoring module 630, for monitoring the storage state of described at least two memory node；
Data recovery module 640, for when described monitoring module monitors the storage state of described first memory node being
During inefficacy, by the number in the internal memory of described first memory node of backup in the persistent storage device of described second memory node
According to, return in the internal memory of described second memory node.
On the basis of above-described embodiment, further, described device also includes:
Acquisition module 650, for being assigned at least two in described distribution module by all resource description framework RDF data
Before the internal memory of memory node stores, obtain the subject of described RDF data, predicate and object；
Memory storage management module 660, for setting up index according to described subject, predicate and object to described RDF data；
Described distribution module 610, for being assigned to the described RDF data establishing index according to concordance hash algorithm
The internal memory of described at least two memory node stores.
On the basis of above-described embodiment, further, described memory storage management module 660, specifically for:
With subject and predicate as key, described RDF data is set up index for value by object；
With subject and object as key, described RDF data is set up index for value by predicate；
With predicate and object as key, described RDF data is set up index for value by subject；
With subject as key, described RDF data is set up index for value by predicate and object；
With predicate as key, described RDF data is set up index for value by subject and predicate；
With object as key, described RDF data is set up index for value by subject and predicate.
The telescopic distributed resource description framework data storage device that the embodiment of the present invention provides is for realizing
Stating the telescopic distributed resource description framework data storage method that embodiment of the method provides, it implements principle and skill
Art effect is similar with embodiment of the method, does not repeats them here.
One of ordinary skill in the art will appreciate that: all or part of step realizing above-mentioned each method embodiment can be led to
The hardware crossing programmed instruction relevant completes.Aforesaid program can be stored in a computer, mobile phone or other portable units
In read/write memory medium.This program upon execution, performs to include the step of above-mentioned each method embodiment；And aforesaid storage is situated between
Matter includes: the various media that can store program code such as ROM, RAM, magnetic disc or CD.
Last it is noted that various embodiments above is only in order to illustrate technical scheme, it is not intended to limit；To the greatest extent
The present invention has been described in detail by pipe with reference to foregoing embodiments, it will be understood by those within the art that: it depends on
So the technical scheme described in foregoing embodiments can be modified, or the most some or all of technical characteristic is entered
Row equivalent；And these amendments or replacement, do not make the essence of appropriate technical solution depart from various embodiments of the present invention technology
The scope of scheme.
Claims (8)
1. a telescopic distributed resource description framework data storage method, it is characterised in that including:
It is assigned in the internal memory of at least two memory node store by all resource description framework RDF data；
Data in the internal memory of each first memory node are backuped in the persistent storage device of the second memory node, described
One memory node and described second memory node are any one memory node in described at least two memory node, described
Two memory nodes are adjacent with described first memory node.
Method the most according to claim 1, it is characterised in that also include:
Monitor the storage state of described at least two memory node；
If the storage state of described first memory node was for losing efficacy, then, by the persistent storage device of described second memory node
Data in the internal memory of described first memory node of backup, return in the internal memory of described second memory node.
Method the most according to claim 1 and 2, it is characterised in that described by the distribution of all resource description framework RDF data
Before storing in the internal memory of at least two memory node, also include:
Obtain the subject of described RDF data, predicate and object；
According to described subject, predicate and object, described RDF data is set up index；
Described being assigned in the internal memory of at least two memory node by all RDF data stores, including:
According to concordance hash algorithm, the described RDF data establishing index is assigned in described at least two memory node
Store in depositing.
Method the most according to claim 3, it is characterised in that described according to described subject, predicate and object to described RDF
Data set up index, including the combination of following any one or more mode:
With subject and predicate as key, described RDF data is set up index for value by object；
With subject and object as key, described RDF data is set up index for value by predicate；
With predicate and object as key, described RDF data is set up index for value by subject；
With subject as key, described RDF data is set up index for value by predicate and object；
With predicate as key, described RDF data is set up index for value by subject and predicate；
With object as key, described RDF data is set up index for value by subject and predicate.
5. a telescopic distributed resource description framework data storage device, it is characterised in that including:
Distribution module, for being assigned in the internal memory of at least two memory node carry out by all resource description framework RDF data
Storage；Described at least two memory node includes the first memory node and the second memory node；
Backup module, for backuping to the lasting of described second memory node by the data in the internal memory of described first memory node
Changing in memorizer, described second memory node is adjacent with described first memory node.
Device the most according to claim 5, it is characterised in that also include:
Monitoring module, for monitoring the storage state of described at least two memory node；
Data recovery module, is used for when described monitoring module monitors the storage state of described first memory node for losing efficacy,
By the data in the internal memory of described first memory node of backup in the persistent storage device of described second memory node, return to
In the internal memory of described second memory node.
7. according to the device described in claim 5 or 6, it is characterised in that also include:
Acquisition module, saves for all resource description framework RDF data being assigned at least two storage in described distribution module
Before the internal memory of point stores, obtain the subject of described RDF data, predicate and object；
Memory storage management module, for setting up index according to described subject, predicate and object to described RDF data；
Described distribution module, described according to concordance hash algorithm the described RDF data establishing index is assigned to extremely
The internal memory of few two memory nodes stores.
Device the most according to claim 7, it is characterised in that described memory storage management module, specifically for:
With subject and predicate as key, described RDF data is set up index for value by object；
With subject and object as key, described RDF data is set up index for value by predicate；
With predicate and object as key, described RDF data is set up index for value by subject；
With subject as key, described RDF data is set up index for value by predicate and object；
With predicate as key, described RDF data is set up index for value by subject and predicate；
With object as key, described RDF data is set up index for value by subject and predicate.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610523135.5A CN106156319A (en) | 2016-07-05 | 2016-07-05 | Telescopic distributed resource description framework data storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610523135.5A CN106156319A (en) | 2016-07-05 | 2016-07-05 | Telescopic distributed resource description framework data storage method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106156319A true CN106156319A (en) | 2016-11-23 |
Family
ID=58061342
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610523135.5A Pending CN106156319A (en) | 2016-07-05 | 2016-07-05 | Telescopic distributed resource description framework data storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106156319A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107122442A (en) * | 2017-04-24 | 2017-09-01 | 上海兴容通信技术有限公司 | A kind of distributed data base and its access method |
| CN107169138A (en) * | 2017-06-13 | 2017-09-15 | 电子科技大学 | A kind of data distributing method of Based on Distributed memory database query engine |
| CN110096515A (en) * | 2019-05-10 | 2019-08-06 | 天津大学深圳研究院 | A kind of RDF data management method, device and storage medium based on triple |
| CN110110034A (en) * | 2019-05-10 | 2019-08-09 | 天津大学深圳研究院 | A kind of RDF data management method, device and storage medium based on figure |
| CN110134335A (en) * | 2019-05-10 | 2019-08-16 | 天津大学深圳研究院 | A kind of RDF data management method, device and storage medium based on key-value pair |
| CN113377577A (en) * | 2021-06-10 | 2021-09-10 | 上海西信信息科技股份有限公司 | Cloud platform architecture based on big data and data processing method thereof |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103425734A (en) * | 2012-02-23 | 2013-12-04 | 富士通株式会社 | Database, apparatus, and method for storing encoded triples |
| US20130332484A1 (en) * | 2012-06-06 | 2013-12-12 | Rackspace Us, Inc. | Data Management and Indexing Across a Distributed Database |
| CN104778277A (en) * | 2015-04-30 | 2015-07-15 | 福州大学 | RDF (radial distribution function) data distributed type storage and querying method based on Redis |
-
2016
- 2016-07-05 CN CN201610523135.5A patent/CN106156319A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103425734A (en) * | 2012-02-23 | 2013-12-04 | 富士通株式会社 | Database, apparatus, and method for storing encoded triples |
| US20130332484A1 (en) * | 2012-06-06 | 2013-12-12 | Rackspace Us, Inc. | Data Management and Indexing Across a Distributed Database |
| CN104778277A (en) * | 2015-04-30 | 2015-07-15 | 福州大学 | RDF (radial distribution function) data distributed type storage and querying method based on Redis |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107122442A (en) * | 2017-04-24 | 2017-09-01 | 上海兴容通信技术有限公司 | A kind of distributed data base and its access method |
| CN107122442B (en) * | 2017-04-24 | 2021-04-16 | 上海兴容信息技术有限公司 | Distributed database and access method thereof |
| CN107169138A (en) * | 2017-06-13 | 2017-09-15 | 电子科技大学 | A kind of data distributing method of Based on Distributed memory database query engine |
| CN107169138B (en) * | 2017-06-13 | 2020-07-14 | 电子科技大学 | Data distribution method for distributed memory database query engine |
| CN110096515A (en) * | 2019-05-10 | 2019-08-06 | 天津大学深圳研究院 | A kind of RDF data management method, device and storage medium based on triple |
| CN110110034A (en) * | 2019-05-10 | 2019-08-09 | 天津大学深圳研究院 | A kind of RDF data management method, device and storage medium based on figure |
| CN110134335A (en) * | 2019-05-10 | 2019-08-16 | 天津大学深圳研究院 | A kind of RDF data management method, device and storage medium based on key-value pair |
| CN110134335B (en) * | 2019-05-10 | 2022-08-12 | 天津大学深圳研究院 | RDF data management method and device based on key value pair and storage medium |
| CN113377577A (en) * | 2021-06-10 | 2021-09-10 | 上海西信信息科技股份有限公司 | Cloud platform architecture based on big data and data processing method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106156319A (en) | Telescopic distributed resource description framework data storage method and device | |
| Koloniari et al. | Peer-to-peer management of XML data: issues and research challenges | |
| US8166074B2 (en) | Index data structure for a peer-to-peer network | |
| CN103020315B (en) | A kind of mass small documents storage means based on master-salve distributed file system | |
| Ji et al. | Big data processing: Big challenges and opportunities | |
| CN104102710A (en) | Massive data query method | |
| Gajendran | A survey on nosql databases | |
| Cambazoglu et al. | Scalability challenges in web search engines | |
| CN104820717A (en) | Massive small file storage and management method and system | |
| CN106569896B (en) | A kind of data distribution and method for parallel processing and system | |
| CN103812939A (en) | Big data storage system | |
| CN103246749A (en) | Matrix data base system for distributed computing and query method thereof | |
| CN104239377A (en) | Platform-crossing data retrieval method and device | |
| Mami et al. | Uniform access to multiform data lakes using semantic technologies | |
| CN103617276A (en) | Method for storing distributed hierarchical RDF data | |
| CN103823846A (en) | Method for storing and querying big data on basis of graph theories | |
| Hu et al. | Towards big linked data: a large-scale, distributed semantic data storage | |
| Von der Weth et al. | Multiterm keyword search in NoSQL systems | |
| CN107682395A (en) | A kind of big data cloud computing runtime and method | |
| Chang et al. | Integration and optimization of multiple big data processing platforms | |
| Abu-Salih et al. | Introduction to big data technology | |
| CN107943412A (en) | A kind of subregion division, the method, apparatus and system for deleting data file in subregion | |
| CN116541427B (en) | Data query method, device, equipment and storage medium | |
| KR101955376B1 (en) | Processing method for a relational query in distributed stream processing engine based on shared-nothing architecture, recording medium and device for performing the method | |
| CN112417225A (en) | Joint query method and system for multi-source heterogeneous data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161123 |
|
| RJ01 | Rejection of invention patent application after publication |