CN105389334A - Conversion method and device of data storage structure - Google Patents
Conversion method and device of data storage structure Download PDFInfo
- Publication number
- CN105389334A CN105389334A CN201510674542.1A CN201510674542A CN105389334A CN 105389334 A CN105389334 A CN 105389334A CN 201510674542 A CN201510674542 A CN 201510674542A CN 105389334 A CN105389334 A CN 105389334A
- Authority
- CN
- China
- Prior art keywords
- data
- chained list
- converted
- key
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
Abstract
The invention provides a conversion scheme of a data storage structure, and comprises the following steps: extracting the position identifier of each piece of data to be converted in the same linked list on the basis of an original linked list storage structure, and the linked list identifier of the linked list, and independently combining the linked list identifier with the position identifier of each piece of data to be converted to determine a key in a conversion storage structure corresponding to each piece of data to be converted; subsequently, combining the extracted position identifiers in the left adjacent data and the right adjacent data of each piece of data to be converted in the linked list with a value data unit in each piece of extracted data to be converted to determine as a value in the conversion storage structure of each piece of data to be converted; and according to the conversion storage structure, storing a data key value formed by the key and the value into a key value database. A storage system which is high in data storage processing efficiency, supports various types of data storage structures and is free from the limitation of a physical memory can be finally realized.
Description
Technical field
The present invention relates to field of computer technology, specifically, the conversion method of a kind of data store organisation that the present invention relates to and device.
Background technology
Along with developing rapidly of information age, not only in large enterprises, even some small enterprises also have accumulated the data of TB magnitude, the tissue of various scale starts to have had the demand of the data of the extensive and multiple data kind of process, and almost reaches capacity in the easily extensible that traditional relevant database (SQL database) stores in data.In order to solve a difficult problem for challenge that large-scale data set and multiple data kind bring and large market demand, non-relational database (NoSQL database) is arisen at the historic moment.
Conventional non-relational database comprises KV (Key-Value, key assignments) stored data base, and its data carry out organizing according to the form of key-value pair, index and storage.KV storage mode is applicable to the operational data storage not relating to too much data relationship and business relations very much, it effectively can reduce the number of times of read-write disk, better readwrite performance is had than SQL database storage, KV stored data base implementation can comprise Redis and LevelDB etc., LevelDB can support the data volume of billion rank, and very high data storage processing performance can also be kept under this quantity rank, simultaneously, data can be stored to disk by LevelDB, are conducive to the persistence of data; But the interface of current LevelDB only supports key-value (key-value pair) the type data without distortion, and interface is comparatively single, poor to the extensibility of business.Redis is a kind of high performance key-value storage system, it can support the data store organisation of multiple value type, comprise string (character string) type, list (chained list) type, set (set) type, zset (ordered set) type and hash (Hash) type, but, in order to ensure higher data storage processing efficiency, its data are all be buffered in internal memory, namely Redis is a kind of internal memory type database, therefore this storage mode limits by physical memory, cannot realize large data and store.
As can be seen here, how to realize a kind of data storage processing efficiency higher, not only supported polytype data store organisation, but also be not current urgent problem by the storage system of physical memory restriction.
Summary of the invention
For overcoming above-mentioned technical matters or solving the problems of the technologies described above at least in part, the following technical scheme of special proposition:
Embodiments of the invention propose a kind of conversion method of data store organisation, comprising:
Extract and identify based on the station location marker of each data to be converted in the same chained list of original chained list storage organization and the chained list of described chained list, and the station location marker of described chained list mark and each data to be converted described is merged, respectively to determine the key in the conversion storage organization corresponding to each data to be converted described;
Extract and the station location marker of each data to be converted described in described chained list in the adjacent two data of left and right and the value data cell in each data to be converted described, and by extract with the station location marker of each data to be converted described in described chained list in the adjacent two data of left and right with extract described in value data cell in each data to be converted merge respectively, to be defined as the value in the conversion storage organization of each data to be converted described;
According to described conversion storage organization, the data key values described key and described value formed is to being stored in key value database.
Preferably, the method also comprises:
Determine structurally interrelated information corresponding to original chained list storage organization in described chained list;
Each data to be converted be designated with predetermined link table information in key, the structure key-value pair formed for value with described structurally interrelated information and described same chained list are stored in key value database explicitly.
Wherein, described structurally interrelated information, at least comprises:
The quantity of the data to be converted in described chained list;
Next available station location marker in described chained list;
The station location marker of the header data of described chained list;
The station location marker of the tail data of described chained list.
Wherein, described predetermined link table information mark, at least comprises:
The chained list mark of described chained list; Variable identifies.
Preferably, the method also comprises:
When to arbitrary data key values to be updated to when carrying out renewal rewards theory, to data key values corresponding to two data adjacent to this data key values to be updated left and right in described chained list to carrying out corresponding renewal
And to this data key values to be updated, corresponding renewal is carried out to the structure key-value pair be associated.
In the present invention, another embodiment also proposed a kind of conversion equipment of data store organisation, it is characterized in that, comprising:
Key determination module, chained list for the station location marker and described chained list that extract each data to be converted in the same chained list based on original chained list storage organization identifies, and the station location marker of described chained list mark and each data to be converted described is merged, respectively to determine the key in the conversion storage organization corresponding to each data to be converted described;
Value determination module, for extracting station location marker in described chained list in the adjacent two data of left and right and the value data cell in each data to be converted described with each data to be converted described, and by extract with the station location marker of each data to be converted described in described chained list in the adjacent two data of left and right with extract described in value data cell in each data to be converted merge respectively, to be defined as the value in the conversion storage organization of each data to be converted described;
Data key values to memory module, for according to described conversion storage organization, by the data key values of described key and described value composition to being stored in key value database.
Preferably, this device also comprises:
Structure determination module, for determining structurally interrelated information corresponding to original chained list storage organization in described chained list;
Structure key-value pair memory module, for being stored to each data to be converted be designated with predetermined link table information in key, the structure key-value pair formed for value with described structurally interrelated information and described same chained list explicitly in key value database.
Wherein, described structurally interrelated information, at least comprises:
The quantity of the data to be converted in described chained list;
Next available station location marker in described chained list;
The station location marker of the header data of described chained list;
The station location marker of the tail data of described chained list.
Wherein, described predetermined link table information mark, at least comprises:
The chained list mark of described chained list; Variable identifies.
Preferably, this device also comprises:
Update module, for
When to arbitrary data key values to be updated to when carrying out renewal rewards theory, to data key values corresponding to two data adjacent to this data key values to be updated left and right in described chained list to carrying out corresponding renewal,
And to this data key values to be updated, corresponding renewal is carried out to the structure key-value pair be associated.
In embodiments of the invention, propose a kind of conversion plan of data store organisation, by the data conversion based on original storage structure is stored as based on the data in conversion storage organization, realize carrying out data storage to multiple storage data type in the mode of unified conversion storage organization.Further, by the data of the several data storage organization in the Redis storage system of the data store organisation of the multiple value type of support, unloading is to the LevelDB storage system only supporting the key-value pair data storage organization without distortion, achieve under the prerequisite ensureing higher data storage processing efficiency, being converted to the storage mode limited by physical memory with disk is storage mode that data store target, that be not subject to physical memory restriction.Pass through the present invention, finally can realize a kind of data storage processing efficiency higher, not only supported polytype data store organisation, but also by the storage system of physical memory restriction, because this storage system can carry out efficient storage to the mass data of plurality of data structures type, thus the data storage method of this programme is made to have stronger operation expanding.
The aspect that the present invention adds and advantage will part provide in the following description, and these will become obvious from the following description, or be recognized by practice of the present invention.
Accompanying drawing explanation
The present invention above-mentioned and/or additional aspect and advantage will become obvious and easy understand from the following description of the accompanying drawings of embodiments, wherein:
Fig. 1 is the process flow diagram of the conversion method of the data store organisation of an embodiment in the present invention;
Fig. 2 is the process flow diagram of the conversion method of the data store organisation of a preferred embodiment in the present invention;
Fig. 3 is the structural representation of the conversion equipment of the data store organisation of another embodiment in the present invention;
Fig. 4 is the structural representation of the device of the conversion of the data store organisation of another preferred embodiment in the present invention;
Fig. 5 is the structural representation of data to be converted in list type chained list;
Fig. 6 is station location marker and the structural representation of each data to be converted in a list type chained list;
Fig. 7 is station location marker and the structural representation of each data after the 3rd Nodes adds a new node in the list type chained list of Fig. 6;
Fig. 8 is station location marker and the structural representation of each data delete Section Point in the list type chained list of Fig. 7 after.
Embodiment
Be described below in detail embodiments of the invention, the example of described embodiment is shown in the drawings, and wherein same or similar label represents same or similar element or has element that is identical or similar functions from start to finish.Being exemplary below by the embodiment be described with reference to the drawings, only for explaining the present invention, and can not limitation of the present invention being interpreted as.
Those skilled in the art of the present technique are appreciated that unless expressly stated, and singulative used herein " ", " one ", " described " and " being somebody's turn to do " also can comprise plural form.Should be further understood that, the wording used in instructions of the present invention " comprises " and refers to there is described feature, integer, step, operation, element and/or assembly, but does not get rid of and exist or add other features one or more, integer, step, operation, element, assembly and/or their group.Should be appreciated that, when we claim element to be " connected " or " coupling " to another element time, it can be directly connected or coupled to other elements, or also can there is intermediary element.In addition, " connection " used herein or " coupling " can comprise wireless connections or wirelessly to couple.Wording "and/or" used herein comprises one or more whole or arbitrary unit listing item be associated and all combinations.
Those skilled in the art of the present technique are appreciated that unless otherwise defined, and all terms used herein (comprising technical term and scientific terminology), have the meaning identical with the general understanding of the those of ordinary skill in field belonging to the present invention.It should also be understood that, those terms defined in such as general dictionary, should be understood to that there is the meaning consistent with the meaning in the context of prior art, unless and by specific definitions as here, otherwise can not explain by idealized or too formal implication.
Fig. 1 is the process flow diagram of the conversion method of the data store organisation of an embodiment in the present invention.
Step S110: extract the station location marker of each data to be converted in the same chained list based on original chained list storage organization and the chained list mark of chained list, and the station location marker of chained list mark and each data to be converted is merged, respectively to determine the key in the conversion storage organization that data to be converted to each are corresponding; Step S120: extraction is with the station location marker of each data to be converted in chained list in the adjacent two data of left and right and the value data cell in each data to be converted and by merging respectively, to be defined as the value in the conversion storage organization of each data to be converted with the station location marker of each data to be converted in chained list in the adjacent two data of left and right and the value data cell in each data to be converted extracted of extracting; Step S130: according to conversion storage organization, data key values key and value formed is to being stored in key value database.
In embodiments of the invention, propose a kind of conversion plan of data store organisation, by the data conversion based on original storage structure is stored as based on the data in conversion storage organization, realize carrying out data storage to multiple storage data type in the mode of unified conversion storage organization.Further, by the data of the several data storage organization in the Redis storage system of the data store organisation of the multiple value type of support, unloading is to the LevelDB storage system only supporting the key-value pair data storage organization without distortion, achieve under the prerequisite ensureing higher data storage processing efficiency, being converted to the storage mode limited by physical memory with disk is storage mode that data store target, that be not subject to physical memory restriction.Pass through the present invention, finally can realize a kind of data storage processing efficiency higher, not only supported polytype data store organisation, but also by the storage system of physical memory restriction, because this storage system can carry out efficient storage to the mass data of plurality of data structures type, thus the data storage method of this programme is made to have stronger operation expanding.
In the present embodiment, for original chained list storage organization for list type chain list data structure, this list type chained list is the doubly linked list structure of band data field, and the structure of data to be converted in chained list as shown in Figure 5.
Wherein, data represents the value data cell in these data to be converted, and prior represents the station location marker of the left side adjacent data of these data to be converted, and next represents the station location marker of the right adjacent data of these data to be converted.
If there are 3 data to be converted, i.e. 3 nodes in list type chained list, the station location marker of each data to be converted and structure are as shown in Figure 6.
Wherein, comprise in the chained list of 3 data to be converted at this, the station location marker of the data to be converted of first node is 1, value data cell is a, and the station location marker of left side adjacent data is-1, and the station location marker of right adjacent data is 2; The station location marker of the data to be converted of Section Point is 2, value data cell is b, and the station location marker of left side adjacent data is 1, and the station location marker of right adjacent data is 3; The station location marker of the data to be converted of the 3rd node is 3, value data cell is c, and the station location marker of left side adjacent data is 2, and the station location marker of right adjacent data is-1; Wherein, because the data to be converted of first node do not exist front nodal point, namely there is not the station location marker of left side adjacent data, then be set to-1, in like manner, there is not posterior nodal point in the data to be converted of the 3rd node,, namely there is not the station location marker of the right adjacent data, be set to-1 yet.
Step S110: extract the station location marker of each data to be converted in the same chained list based on original chained list storage organization and the chained list mark of chained list, and the station location marker of chained list mark and each data to be converted is merged, respectively to determine the key in the conversion storage organization that data to be converted to each are corresponding.
Such as, extract if the station location marker 1,2,3 of each data to be converted in the list type chain list data structure of Fig. 6 and the chained list mark of this chained list are as K, and the station location marker of chained list mark and each data to be converted is merged respectively, to determine the key in the conversion storage organization that data to be converted to each are corresponding, if the key obtaining the data to be converted of first node is K+1, the key of the data to be converted of Section Point is K+2, and the key of the data to be converted of the 3rd node is K+3.
Step S120: extract and the station location marker of each data to be converted in chained list in the adjacent two data of left and right and the value data cell in each data to be converted, and by merging respectively, to be defined as the value in the conversion storage organization of each data to be converted with the station location marker of each data to be converted in chained list in the adjacent two data of left and right and the value data cell in each data to be converted extracted of extracting.
Such as, connect example, by extract with the station location marker of each data to be converted in chained list in the adjacent two data of left and right, prior as the data to be converted of first node is-1, next is 2, merge with the value data cell a of the data to be converted as first node extracted, to be defined as the value in the conversion storage organization of the data to be converted of first node, the value namely obtaining the data to be converted of first node is "-1,2, a "; In like manner, the value of the data to be converted of Section Point is "-1,3, b ", and the value of the data to be converted of the 3rd node is " 2 ,-1, c ".
Step S130: according to conversion storage organization, data key values key and value formed is to being stored in key value database.
Such as, connect example, key and value are merged composition data key values pair, and data key values is stored in key value database " K+1-'-1,2, a ' ", " K+2-'-1,3, b ' " and " K+3-' 2 ,-1, c ' ".
In a preferred embodiment, as shown in Figure 2, the method also comprises step S210 and step S220; Step S210: determine structurally interrelated information corresponding to original chained list storage organization in chained list; Step S220: by being designated key with predetermined link table information, be that each data to be converted in the structure key-value pair of value composition and same chained list are stored in key value database explicitly with structurally interrelated information.
Wherein, structurally interrelated information specifically includes but not limited to:
The quantity of the data to be converted in chained list;
Next available station location marker in chained list;
The station location marker of the header data of chained list;
The station location marker of the tail data of chained list.
Wherein, predetermined link table information mark, at least comprises:
The chained list mark of chained list; Variable identifies.Chained list as being chained list with " K " identifies, and " meta " is variable mark.
Such as, connect example, structurally interrelated information corresponding to the storage of linked list structure shown in Fig. 6 in chained list comprises: as next available station location marker, the station location marker of header data of chained list and the station location marker of the tail data of chained list in the quantity of the data to be converted in chained list, chained list, according to Fig. 6, can determine that the quantity of the data to be converted in chained list is 3, next available station location marker in chained list is 4, the station location marker of the header data of chained list is 1, the station location marker of the tail data of chained list is 3, with predetermined link table information mark " K
meta" be key, structurally interrelated information " 3,4,1,3 " is the structure key-value pair " K of value composition
meta-' 3,4,1,3 ' " be stored to explicitly in key value database with each data to be converted in this chained list.Wherein, structure key-value pair and each data to be converted explicitly mode are, the key " K of structure key-value pair
meta" comprise for element " K "; the data key values of each data to be converted is to including as element " K " in each key " K+1 " of " K+1-'-1,2, a ' ", " K+2-'-1,3, b ' " and " K+3-' 2 ,-1, c ' ", " K+2 " and " K+3 ", and it can be associated relation by " K ".
Preferably, the method also comprises:
When to arbitrary data key values to be updated to when carrying out renewal rewards theory, to data key values corresponding to two data adjacent to this data key values to be updated left and right in described chained list to carrying out corresponding renewal;
When to arbitrary data key values to be updated to when carrying out renewal rewards theory, to this data key values to be updated, corresponding renewal is carried out to the structure key-value pair be associated.
Such as, connect example, a new node is added in Section Point in the chained list shown in Fig. 6 and the 3rd node, value data cell is d, the then station location marker of the prior of Section Point, can change with the station location marker of the next of the 3rd node thereupon, the Section Point data key values corresponding to the 3rd node carries out corresponding renewal to also needing, the data key values of former Section Point is to being updated to " K+2-' 1,4, b ' ", and the data key values of former 3rd node is to being updated to " K+3-' 4 ,-1, c ' "; The corresponding structure key-value pair of this chained list also there occurs change, the quantity increasing the data to be converted in the chained list after new node is 4, next available station location marker in chained list is 5, the station location marker of the header data of chained list is 1, the station location marker of the tail data of chained list is 3, with predetermined link table information mark " K
meta" be key, structurally interrelated information " 4,5,1,3 ", therefore corresponding structure key-value pair is updated to " K
meta-' 4,5,1,3 ' ", the list structure after adding is obtained as shown in Figure 7.
Again such as, connect example, the Section Point that value data cell is b is deleted in Fig. 7 chained list, then next available station location marker of first node and the last available station location marker of the 3rd node there occurs change, corresponding data key values upgrades accordingly to also needing, now the data key values of former first node is to being updated to " K+1 – '-1,4, a ' ", and the data key values of former 3rd node is to being updated to " K+4-' 1,3, d ' "; Corresponding structure key-value pair also there occurs change, the quantity of deleting the data to be converted in the chained list after Section Point is 3, next available station location marker in chained list is 5, the station location marker of the header data of chained list is 1, the station location marker of the tail data of chained list is 3, with predetermined link table information mark " K
meta" be key, structurally interrelated information " 3,5,1,3 ", therefore corresponding structure key-value pair is updated to " K
meta-' 3,5,1,3 ' ", the list structure after deleting is obtained as shown in Figure 8.
Fig. 3 is the structural representation of the conversion equipment of the data store organisation of another embodiment in the present invention.
Key determination module 310 extracts the station location marker of each data to be converted in the same chained list based on original chained list storage organization and the chained list mark of chained list, and the station location marker of chained list mark and each data to be converted is merged, respectively to determine the key in the conversion storage organization that data to be converted to each are corresponding; Value determination module 320 extraction is with the station location marker of each data to be converted in chained list in the adjacent two data of left and right and the value data cell in each data to be converted and by merging respectively, to be defined as the value in the conversion storage organization of each data to be converted with the station location marker of each data to be converted in chained list in the adjacent two data of left and right and the value data cell in each data to be converted extracted of extracting; Data key values is to memory module 330 according to conversion storage organization, and data key values key and value formed is to being stored in key value database.
In embodiments of the invention, propose a kind of conversion plan of data store organisation, by the data conversion based on original storage structure is stored as based on the data in conversion storage organization, realize carrying out data storage to multiple storage data type in the mode of unified conversion storage organization.Further, by the data of the several data storage organization in the Redis storage system of the data store organisation of the multiple value type of support, unloading is to the LevelDB storage system only supporting the key-value pair data storage organization without distortion, achieve under the prerequisite ensureing higher data storage processing efficiency, being converted to the storage mode limited by physical memory with disk is storage mode that data store target, that be not subject to physical memory restriction.Pass through the present invention, finally can realize a kind of data storage processing efficiency higher, not only supported polytype data store organisation, but also by the storage system of physical memory restriction, because this storage system can carry out efficient storage to the mass data of plurality of data structures type, thus the data storage method of this programme is made to have stronger operation expanding.
In the present embodiment, for original chained list storage organization for list type chain list data structure, this list type chained list is the doubly linked list structure of band data field, and the structure of data to be converted in chained list as shown in Figure 5.
Wherein, data represents the value data cell in these data to be converted, and prior represents the station location marker of the left side adjacent data of these data to be converted, and next represents the station location marker of the right adjacent data of these data to be converted.
If there are 3 data to be converted, i.e. 3 nodes in list type chained list, the station location marker of each data to be converted and structure are as Fig. 6.
Wherein, comprise in the chained list of 3 data to be converted at this, the station location marker of the data to be converted of first node is 1, value data cell is a, and the station location marker of left side adjacent data is-1, and the station location marker of right adjacent data is 2; The station location marker of the data to be converted of Section Point is 2, value data cell is b, and the station location marker of left side adjacent data is 1, and the station location marker of right adjacent data is 3; The station location marker of the data to be converted of the 3rd node is 3, value data cell is c, and the station location marker of left side adjacent data is 2, and the station location marker of right adjacent data is-1; Wherein, because the data to be converted of first node do not exist front nodal point, namely there is not the station location marker of left side adjacent data, then be set to-1, in like manner, there is not posterior nodal point in the data to be converted of the 3rd node,, namely there is not the station location marker of the right adjacent data, be set to-1 yet.
Key determination module 310 extracts the station location marker of each data to be converted in the same chained list based on original chained list storage organization and the chained list mark of chained list, and the station location marker of chained list mark and each data to be converted is merged, respectively to determine the key in the conversion storage organization that data to be converted to each are corresponding.
Such as, extract if the station location marker 1,2,3 of each data to be converted in the list type chain list data structure of Fig. 6 and the chained list mark of this chained list are as K, and the station location marker of chained list mark and each data to be converted is merged respectively, to determine the key in the conversion storage organization that data to be converted to each are corresponding, if the key obtaining the data to be converted of first node is K+1, the key of the data to be converted of Section Point is K+2, and the key of the data to be converted of the 3rd node is K+3.
Value determination module 320 extraction is with the station location marker of each data to be converted in chained list in the adjacent two data of left and right and the value data cell in each data to be converted and by merging respectively, to be defined as the value in the conversion storage organization of each data to be converted with the station location marker of each data to be converted in chained list in the adjacent two data of left and right and the value data cell in each data to be converted extracted of extracting.
Such as, connect example, by extract with the station location marker of each data to be converted in chained list in the adjacent two data of left and right, prior as the data to be converted of first node is-1, next is 2, merge with the value data cell a of the data to be converted as first node extracted, to be defined as the value in the conversion storage organization of the data to be converted of first node, the value namely obtaining the data to be converted of first node is "-1,2, a "; In like manner, the value of the data to be converted of Section Point is "-1,3, b ", and the value of the data to be converted of the 3rd node is " 2 ,-1, c ".
Data key values is to memory module 330 according to conversion storage organization, and data key values key and value formed is to being stored in key value database.
Such as, connect example, key and value are merged composition data key values pair, and data key values is stored in key value database " K+1-'-1,2, a ' ", " K+2-'-1,3, b ' " and " K+3-' 2 ,-1, c ' ".
In a preferred embodiment, as shown in Figure 4, this device also comprises structure determination module 410 and structure key-value pair memory module 420; Structure determination module 410 determines structurally interrelated information corresponding to original chained list storage organization in chained list; Structure key-value pair memory module 420 by being designated key with predetermined link table information, be that each data to be converted in the structure key-value pair of value composition and same chained list are stored in key value database explicitly with structurally interrelated information.
Wherein, structurally interrelated information specifically includes but not limited to:
The quantity of the data to be converted in chained list;
Next available station location marker in chained list;
The station location marker of the header data of chained list;
The station location marker of the tail data of chained list.
Wherein, predetermined link table information mark, at least comprises:
The chained list mark of chained list; Variable identifies.Chained list as being chained list with " K " identifies, and " meta " is variable mark.
Such as, connect example, structurally interrelated information corresponding to the storage of linked list structure shown in Fig. 6 in chained list comprises: as next available station location marker, the station location marker of header data of chained list and the station location marker of the tail data of chained list in the quantity of the data to be converted in chained list, chained list, according to Fig. 6, can determine that the quantity of the data to be converted in chained list is 3, next available station location marker in chained list is 4, the station location marker of the header data of chained list is 1, the station location marker of the tail data of chained list is 3, with predetermined link table information mark " K
meta" be key, structurally interrelated information " 3,4,1,3 " is the structure key-value pair " K of value composition
meta-' 3,4,1,3 ' " be stored to explicitly in key value database with each data to be converted in this chained list.Wherein, structure key-value pair and each data to be converted explicitly mode are, the key " K of structure key-value pair
meta" comprise for element " K "; the data key values of each data to be converted is to including as element " K " in each key " K+1 " of " K+1-'-1,2, a ' ", " K+2-'-1,3, b ' " and " K+3-' 2 ,-1, c ' ", " K+2 " and " K+3 ", and it can be associated relation by " K ".
Preferably, this device also comprises update module (not shown); When to arbitrary data key values to be updated to when carrying out renewal rewards theory, update module:
To data key values corresponding to two data adjacent to this data key values to be updated left and right in described chained list to carrying out corresponding renewal; And to this data key values to be updated, corresponding renewal is carried out to the structure key-value pair be associated.
Such as, connect example, a new node is added in Section Point in the chained list shown in Fig. 6 and the 3rd node, value data cell is d, the then station location marker of the prior of Section Point, can change with the station location marker of the next of the 3rd node thereupon, the Section Point data key values corresponding to the 3rd node carries out corresponding renewal to also needing, the data key values of former Section Point is to being updated to " K+2-' 1,4, b ' ", and the data key values of former 3rd node is to being updated to " K+3-' 4 ,-1, c ' "; The corresponding structure key-value pair of this chained list also there occurs change, the quantity increasing the data to be converted in the chained list after new node is 4, next available station location marker in chained list is 5, the station location marker of the header data of chained list is 1, the station location marker of the tail data of chained list is 3, with predetermined link table information mark " K
meta" be key, structurally interrelated information " 4,5,1,3 ", therefore corresponding structure key-value pair is updated to " K
meta-' 4,5,1,3 ' ", the list structure after adding is obtained as Fig. 7.
Again such as, connect example, the Section Point that value data cell is b is deleted in Fig. 7 chained list, then next available station location marker of first node and the last available station location marker of the 3rd node there occurs change, corresponding data key values upgrades accordingly to also needing, now the data key values of former first node is to being updated to " K+1 – '-1,4, a ' ", and the data key values of former 3rd node is to being updated to " K+4-' 1,3, d ' "; Corresponding structure key-value pair also there occurs change, the quantity of deleting the data to be converted in the chained list after Section Point is 3, next available station location marker in chained list is 5, the station location marker of the header data of chained list is 1, the station location marker of the tail data of chained list is 3, with predetermined link table information mark " K
meta" be key, structurally interrelated information " 3,5,1,3 ", therefore corresponding structure key-value pair is updated to " K
meta-' 3,5,1,3 ' ", the list structure after deleting is obtained as shown in Figure 8.
Those skilled in the art of the present technique are appreciated that the one or more equipment that the present invention includes and relate to for performing in operation described in the application.These equipment for required object and specialized designs and manufacture, or also can comprise the known device in multi-purpose computer.These equipment have storage computer program within it, and these computer programs optionally activate or reconstruct.Such computer program can be stored in equipment (such as, computing machine) in computer-readable recording medium or be stored in and be suitable for store electrons instruction and be coupled in the medium of any type of bus respectively, described computer-readable medium includes but not limited to that the dish of any type (comprises floppy disk, hard disk, CD, CD-ROM, and magneto-optic disk), ROM (Read-OnlyMemory, ROM (read-only memory)), RAM (RandomAccessMemory, storer immediately), EPROM (ErasableProgrammableRead-OnlyMemory, Erarable Programmable Read only Memory), EEPROM (ElectricallyErasableProgrammableRead-OnlyMemory, EEPROM (Electrically Erasable Programmable Read Only Memo)), flash memory, magnetic card or light card.Namely, computer-readable recording medium comprises and being stored or any medium of transmission information with the form that can read by equipment (such as, computing machine).
Those skilled in the art of the present technique are appreciated that the combination that can realize the frame in each frame in these structural drawing and/or block diagram and/or flow graph and these structural drawing and/or block diagram and/or flow graph with computer program instructions.Those skilled in the art of the present technique are appreciated that, the processor that these computer program instructions can be supplied to multi-purpose computer, special purpose computer or other programmable data disposal routes realizes, thus is performed the scheme of specifying in the frame of structural drawing disclosed by the invention and/or block diagram and/or flow graph or multiple frame by the processor of computing machine or other programmable data disposal routes.
Those skilled in the art of the present technique are appreciated that various operations, method, the step in flow process, measure, the scheme discussed in the present invention can be replaced, changes, combines or delete.Further, there is various operations, method, other steps in flow process, measure, the scheme discussed in the present invention also can be replaced, change, reset, decompose, combine or delete.Further, of the prior art have also can be replaced with the step in operation various disclosed in the present invention, method, flow process, measure, scheme, changed, reset, decomposed, combined or deleted.
The above is only some embodiments of the present invention; it should be pointed out that for those skilled in the art, under the premise without departing from the principles of the invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.
Claims (10)
1. a conversion method for data store organisation, is characterized in that, comprising:
Extract and identify based on the station location marker of each data to be converted in the same chained list of original chained list storage organization and the chained list of described chained list, and the station location marker of described chained list mark and each data to be converted described is merged, respectively to determine the key in the conversion storage organization corresponding to each data to be converted described;
Extract and the station location marker of each data to be converted described in described chained list in the adjacent two data of left and right and the value data cell in each data to be converted described, and by extract with the station location marker of each data to be converted described in described chained list in the adjacent two data of left and right with extract described in value data cell in each data to be converted merge respectively, to be defined as the value in the conversion storage organization of each data to be converted described;
According to described conversion storage organization, the data key values described key and described value formed is to being stored in key value database.
2. the conversion method of data store organisation according to claim 1, wherein, the method also comprises:
Determine structurally interrelated information corresponding to original chained list storage organization in described chained list;
Each data to be converted be designated with predetermined link table information in key, the structure key-value pair formed for value with described structurally interrelated information and described same chained list are stored in key value database explicitly.
3. the conversion method of data store organisation according to claim 2, wherein, described structurally interrelated information at least comprises:
The quantity of the data to be converted in described chained list;
Next available station location marker in described chained list;
The station location marker of the header data of described chained list;
The station location marker of the tail data of described chained list.
4. the conversion method of the data store organisation according to Claims 2 or 3, wherein, described predetermined link table information mark at least comprises:
The chained list mark of described chained list; Variable identifies.
5. the conversion method of data store organisation according to claim 1, wherein, the method also comprises:
When to arbitrary data key values to be updated to when carrying out renewal rewards theory, to data key values corresponding to two data adjacent to this data key values to be updated left and right in described chained list to carrying out corresponding renewal,
And to this data key values to be updated, corresponding renewal is carried out to the structure key-value pair be associated.
6. a conversion equipment for data store organisation, is characterized in that, comprising:
Key determination module, chained list for the station location marker and described chained list that extract each data to be converted in the same chained list based on original chained list storage organization identifies, and the station location marker of described chained list mark and each data to be converted described is merged, respectively to determine the key in the conversion storage organization corresponding to each data to be converted described;
Value determination module, for extracting station location marker in described chained list in the adjacent two data of left and right and the value data cell in each data to be converted described with each data to be converted described, and by extract with the station location marker of each data to be converted described in described chained list in the adjacent two data of left and right with extract described in value data cell in each data to be converted merge respectively, to be defined as the value in the conversion storage organization of each data to be converted described;
Data key values to memory module, for according to described conversion storage organization, by the data key values of described key and described value composition to being stored in key value database.
7. the conversion equipment of data store organisation according to claim 6, wherein, this device also comprises:
Structure determination module, for determining structurally interrelated information corresponding to original chained list storage organization in described chained list;
Structure key-value pair memory module, for being stored to each data to be converted be designated with predetermined link table information in key, the structure key-value pair formed for value with described structurally interrelated information and described same chained list explicitly in key value database.
8. the conversion equipment of data store organisation according to claim 7, wherein, described structurally interrelated information at least comprises:
The quantity of the data to be converted in described chained list;
Next available station location marker in described chained list;
The station location marker of the header data of described chained list;
The station location marker of the tail data of described chained list.
9. the conversion equipment of the data store organisation according to claim 7 or 8, wherein, described predetermined link table information mark at least comprises:
The chained list mark of described chained list; Variable identifies.
10. the conversion equipment of data store organisation according to claim 9, wherein, this device also comprises update module:
Update module, for when to arbitrary data key values to be updated to when carrying out renewal rewards theory, to data key values corresponding to two data adjacent to this data key values to be updated left and right in described chained list to carrying out corresponding renewal, and to this data key values to be updated, corresponding renewal is carried out to the structure key-value pair be associated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510674542.1A CN105389334B (en) | 2015-10-16 | 2015-10-16 | The conversion method and device of data store organisation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510674542.1A CN105389334B (en) | 2015-10-16 | 2015-10-16 | The conversion method and device of data store organisation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105389334A true CN105389334A (en) | 2016-03-09 |
| CN105389334B CN105389334B (en) | 2019-03-01 |
Family
ID=55421624
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510674542.1A Active CN105389334B (en) | 2015-10-16 | 2015-10-16 | The conversion method and device of data store organisation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105389334B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106777293A (en) * | 2016-12-29 | 2017-05-31 | 北京奇虎科技有限公司 | Data storage and lookup method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080027980A1 (en) * | 2003-10-21 | 2008-01-31 | United Parcel Service Of America, Inc. | Data Structure And Management System For A Superset Of Relational Databases |
| US20090313222A1 (en) * | 2008-06-11 | 2009-12-17 | Nintendo Co., Ltd. | Computer readable storage medium having data processing program recorded thereon and data processing apparatus |
| CN104216961A (en) * | 2014-08-21 | 2014-12-17 | 北京奇艺世纪科技有限公司 | Method and device for data processing |
-
2015
- 2015-10-16 CN CN201510674542.1A patent/CN105389334B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080027980A1 (en) * | 2003-10-21 | 2008-01-31 | United Parcel Service Of America, Inc. | Data Structure And Management System For A Superset Of Relational Databases |
| US20090313222A1 (en) * | 2008-06-11 | 2009-12-17 | Nintendo Co., Ltd. | Computer readable storage medium having data processing program recorded thereon and data processing apparatus |
| CN104216961A (en) * | 2014-08-21 | 2014-12-17 | 北京奇艺世纪科技有限公司 | Method and device for data processing |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106777293A (en) * | 2016-12-29 | 2017-05-31 | 北京奇虎科技有限公司 | Data storage and lookup method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105389334B (en) | 2019-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105260464A (en) | Data storage structure conversion method and apparatus | |
| CN110879813B (en) | Binary log analysis-based MySQL database increment synchronization implementation method | |
| CN111046034B (en) | Method and system for managing memory data and maintaining data in memory | |
| CN106462592B (en) | System and method for optimizing multi-version support for indexes | |
| CN103514201B (en) | Method and device for querying data in non-relational database | |
| US20160314161A1 (en) | Multi-Version Concurrency Control Method in Database and Database System | |
| CN106933836B (en) | Data storage method and system based on sub-tables | |
| CN109901786B (en) | Data migration method, system, device and computer readable storage medium | |
| CN107977396B (en) | Method and device for updating data table of KeyValue database | |
| CN103678491A (en) | Method based on Hadoop small file optimization and reverse index establishment | |
| US10891202B2 (en) | Recovery of in-memory databases using a backward scan of the database transaction log | |
| CN102314485A (en) | Method and device for adding, searching and deleting hash table | |
| CN105138622B (en) | For the insertion operation of LSM tree storage systems and reading and the merging method of load | |
| CN104298736A (en) | Method and device for aggregating and connecting data as well as database system | |
| CN103064908B (en) | A kind of method by the quick duplicate removal list of internal memory | |
| CN108984686A (en) | A kind of distributed file system indexing means and device merged based on log | |
| CN108170752A (en) | metadata management method and system based on template | |
| CN105302889A (en) | Conversion method and apparatus for data storage structure | |
| CN105787090A (en) | Index building method and system of OLAP system of electric data | |
| CN106095850A (en) | A kind of data processing method and equipment | |
| EP3955256A1 (en) | Non-redundant gene clustering method and system, and electronic device | |
| CN113094442B (en) | Full data synchronization method, device, equipment and medium | |
| CN103984723A (en) | Method used for updating data mining for frequent item by incremental data | |
| CN105389334A (en) | Conversion method and device of data storage structure | |
| CN111176901B (en) | HDFS deleted file recovery method, terminal device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220718 Address after: Room 801, 8th floor, No. 104, floors 1-19, building 2, yard 6, Jiuxianqiao Road, Chaoyang District, Beijing 100015 Patentee after: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Address before: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park) Patentee before: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Patentee before: Qizhi software (Beijing) Co.,Ltd. |
|
| TR01 | Transfer of patent right |