CN103473239B - A kind of data of non relational database update method and device - Google Patents
A kind of data of non relational database update method and device Download PDFInfo
- Publication number
- CN103473239B CN103473239B CN201210187686.0A CN201210187686A CN103473239B CN 103473239 B CN103473239 B CN 103473239B CN 201210187686 A CN201210187686 A CN 201210187686A CN 103473239 B CN103473239 B CN 103473239B
- Authority
- CN
- China
- Prior art keywords
- file
- data
- data file
- new data
- new
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention discloses a kind of data of non relational database update method and device, the method includes: receives and updates instruction, and records the described more new data updating and carrying in instruction, and described more new data packets includes major key and corresponding renewal operation content；When the first default trigger event occurs, search the data file belonging to major key in each more new data；According to the described renewal operation content in more new data, data file corresponding for each more new data is merged in new data file, and replaces the data file before merging by described new data file.The method using the present invention to provide, can improve the efficiency that non-relational database historical data updates.
Description
Technical field
The present invention relates to networking technology area, particularly to a kind of data of non relational database update method and device.
Background technology
Table in relevant database all has fixing data structure, each tuple (each note in database table
Record) field groups Chengdu the same, even if not being that each tuple is required for all of field, but data base can be for the institute of each tuple
There are the corresponding data of field distribution.Such as, certain database table includes name, age, three, native place field, then in this table
Every record is required for including name, age, three, native place field, even if wherein the content of certain field of certain tuple is
Sky, system also to distribute a default value for it, such as 0.Such structure can so that the operation such as being attached between table and table,
But for another angle, it is also the key factor causing relevant database performance bottleneck so that relational data
Storehouse can not meet the demand to data base's height concurrent reading and writing, stores the high efficiency of mass data and the demand of access, to data
The enhanced scalability in storehouse and the demand of high availability.
In order to solve the problems referred to above, non-relational database (NoSQL) is arisen at the historic moment.The BigTable of Google with
The Dynamo of Amazon is very successful business NoSQL system.The NoSQL system that some are increased income, such as Membase,
MongoDB, Cassandra, BeansDB, Redis etc., have also been obtained extensively approval.
Non-relational database, it stores with key-value pair, and structure do not fixes, and each tuple can have different word
Section, each tuple can increase some oneself key-value pair as required, be not limited to the structure fixed, it is possible to reduce some time
Between and the expense in space.KV(Key-Value, key assignments) storage, it is a kind of mode of NoSQL storage, data are according to key-value pair
Form carries out organizing, index and storing.KV storage is especially suitable for being not related to the business datum of too much data relationship and business relations,
Can effectively reduce the number of times of read-write disk simultaneously, have more preferable readwrite performance than relational data library storage.
Key assignments memory mechanism uses the storage of key-value pair form, and value can be any random length data, uses major key can make letter
Single inquiry and complex query (such as list query, range query etc.), be periodically updated the merging of data and historical data simultaneously
Operations such as () this union operation can include adding, deletes, amendments is to process stale data.As it is shown in figure 1, more new data and going through
History data acquisition, with 0,1 directory management, arranges 0,1 two catalogues for more new data, and historical data is also provided with 0,1 two mesh
Record, it is assumed that current renewal data directory and historical data catalogue are all 0 catalogue, in the conjunction being updated data and historical data
And time, 0 catalogue of 0 catalogue of more new data and historical data is merged, and the data after merging are stored in the 1 of historical data
In catalogue, simultaneously in merging process, new more new data is written in 1 catalogue of more new data.Once it is updated upper
During the merging of data and historical data, 1 catalogue of more new data and 1 catalogue of historical data are merged into 0 mesh of historical data
In record, the rest may be inferred.
However, it is found by the inventors that, at least there are the following problems for prior art:
In existing data-updating method, the most more new data and the merging of historical data, will be by all of history number
Merging according to more new data, the historical data the most not changed also to merge and to move in another catalogue, effect
Rate is relatively low, considerably increases the expense of system.
Summary of the invention
It is an object of the invention to provide a kind of data of non relational database update method and device, to improve non-relation
The efficiency that type data historian data update, to this end, the embodiment of the present invention adopts the following technical scheme that
A kind of data of non relational database update method, including:
Receiving and update instruction, and record the described more new data updating and carrying in instruction, described more new data packets includes major key
With corresponding renewal operation content；
When the first default trigger event occurs, search the data file belonging to major key in each more new data；
According to the described renewal operation content in more new data, data file corresponding for each more new data is merged into
In new data file, and replace the data file before merging by described new data file.
A kind of data of non relational database updating device, including:
Logging modle, is used for receiving renewal instruction, and records the described more new data updating and carrying in instruction, described renewal
Data include major key and corresponding renewal operation content；
Search module, for when the first default trigger event occurs, searching belonging to the major key in each more new data
Data file；
More new module, for according to the described renewal operation content in more new data, by corresponding for each more new data
Data file is merged in new data file, and replaces the data file before merging by described new data file.
The above embodiment of the present invention, receives and updates instruction, and records the more new data carried in renewal instruction, updates number
According to including major key and corresponding renewal operation content, when the first default trigger event occurs, that searches record respectively updates number
The data file belonging to major key according to, according to the described renewal operation content in more new data, by right with it for the most more new data
The data file answered is merged in new data file, and replaces the data file before merging by described new data file, from
And, the efficiency that non-relational database historical data updates can be improved.
Accompanying drawing explanation
Fig. 1 is the storage mode schematic diagram of more new data and historical data in prior art；
The structural representation of SSTable in the data of non relational database update method that Fig. 2 provides for the embodiment of the present invention
Figure；
The schematic flow sheet of the data of non relational database update method that Fig. 3 provides for the embodiment of the present invention；
According to updating data search pair in the data of non relational database update method that Fig. 4 provides for the embodiment of the present invention
Answer the schematic diagram of data file；
Meta data file storage content in the data of non relational database update method that Fig. 5 provides for the embodiment of the present invention
Schematic diagram；
The flow process of data query method in the data of non relational database update method that Fig. 6 provides for the embodiment of the present invention
Schematic diagram；
The schematic flow sheet of the data of non relational database update method that Fig. 7 provides for the embodiment of the present invention；
The structural representation of the data of non relational database updating device that Fig. 8 provides for the embodiment of the present invention.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the present invention, the technical scheme in the present invention is carried out clear, complete description, aobvious
So, described embodiment is a part of embodiment of the present invention rather than whole embodiments.Based on the enforcement in the present invention
Example, all other embodiments that those of ordinary skill in the art are obtained on the premise of not making creative work, all belong to
In the scope of protection of the invention.
Embodiment one
In the present embodiment, more new data with the storage organization of Memtable be stored in renewal file in, historical data with
The storage organization of SSTable is stored in one or more data file.SSTable structure is the storage of a kind of distributed key assignments
Structure, can comprise multiple SSTable in each data file, can store multiple value(master in each SSTable
The value that key is corresponding) and the interior index of block of correspondence, the structure of each SSTable can be as shown in Figure 2.Inquiring about when,
Can be first according to key(major key) inquire about its affiliated data file, the most further inquire about it belonging in this data file
SSTable, owing to the quantity of historical data is the hugest, use SSTable structure can promote search efficiency.Memtable
It is the storage organization of a kind of non-distributed, owing to the general quantity of more new data is fewer, so using Memtable structure permissible
Promote storage efficiency.
As it is shown on figure 3, the flow process of process of the data of non relational database update method provided for the embodiment of the present invention,
Specifically include following steps:
Step 301, system receives and updates instruction, and records the more new data carried in renewal instruction, and more new data packets includes
Major key and corresponding renewal operation content.
The attendant of data base can send renewal instruction by server or management terminal, and system receives renewal and refers to
After order, can first the more new data updated in instruction be stored in the middle of renewal file, specifically can deposit to use the form of list
Storage key and corresponding renewal operation content.More new data can include the key that its needs update, and include accordingly
The key-value operation (i.e. update operation content) to carrying out, this renewal operation content can be delete key-value to, add
Add key-value to or amendment value etc..
Step 302, when the first default trigger event occurs, the number belonging to major key in system searching the most more new data
According to file.
Concrete, this first trigger event preset can be to reach to preset more new data to merge cycle, i.e. system in advance
More new data is set and merges the cycle, be updated merging of data and historical data according to the cycle.It addition, the first triggering preset
Event can also is that receiving more new data merges instruction, i.e. can be sent the merging of more new data by database maintenance personnel and refer to
Order, to indicate system to be merged with historical data by more new data.
Concrete, as shown in Figure 4, system can search each more new data according to the metadata of record in meta data file
In data file corresponding for key.Metadata is a kind of data for describing data, the metadata of record in meta data file
Queue can be as it is shown in figure 5, the storage of the scope of key, file in the reference number of a document of each data file, file can be included
The information such as capacity.It addition, can be with the information of log file numbering in meta data file, as with reference number of a document, available literary composition
Part is numbered.
Step 303, system operates content according to the renewal in more new data, by data literary composition corresponding for each more new data
Part is merged in new data file, and replaces the data file before merging by new data file.
System, for every more new data, finds the data file of its correspondence according to the key in more new data, then will more
New data merges with corresponding data file.Concrete, the operation of merging is according in the renewal operation in more new data
Hold and carry out, can include deleting key-value to, add key-value to or amendment value etc..
When multiple more new data key correspond to same data file, these multiple more new datas are common and these data are civilian
Part is merged in new data file.Merging process can be according to each key and value in former data file, and more new data
In key and corresponding renewal operation content, according to the order of key, be again sequentially generated each in new data file
The memory capacity of SSTable, SSTable is the numerical value pre-set.
Preferably, system can be to merge the new data file obtained to distribute new reference number of a document (in file available volume
Choose in number).After merging terminates, system can delete the data file being replaced, and the file discharging this data file is compiled
Number, this document numbering is from becoming file available numbering with reference number of a document.
In the above-mentioned flow process of the present embodiment, system can pre-build two for the catalogue recording more new data, will connect
The more new data carried in instruction that updates received recorded in one of them catalogue；Then, occur whenever the first trigger event
Time, carry out catalogue switching, the subsequently received more new data carried in instruction that updates be recorded in another catalogue, and will
In the catalogue before used, each more new data of record is merged in new data file with corresponding data file；To make before
Catalogue in record each more new data be merged in new data file with corresponding data file after, make before deletion
Catalogue in more new data.
Concrete, in the way of using 0,1 catalogue, more new data can be stored, set up more new data respectively
0 catalogue file, 1 catalogue file, when the first trigger event occurs, carry out catalogue switching, it is assumed that currently used is 0 mesh
Record, when the first trigger event proceeds by merging process, the more new data being newly received will be stored in catalogue 1,
More new data in catalogue 0 is merged with corresponding data file, and the renewal after merging process terminates, in 0 catalogue
Data will be deleted, then, when there is the first trigger event next time, then switch back to 0 catalogue be updated data storage, will
More new data in catalogue 1 merges with corresponding data file, and after merging process terminates, and deletes in 1 catalogue more
New data, the rest may be inferred.
Preferably, the embodiment of the present invention additionally provide data of non relational database update method, it is also possible to including:
When the second default trigger event occurs, it is judged that whether the memory capacity sum of adjacent multiple data files exceedes is preset storage
Capacity threshold, without exceeding, then merges into a data file by the plurality of data file.Therein second triggers thing
Part can be to reach the preset data Piece file mergence cycle, it is also possible to is that each more new data merges complete with corresponding data file,
Can also is that receiving data file merges instruction.The merging process of concrete data file may include steps of:
Step A, is ranked up data file according to the order of the major key comprised in data file, by the data after sequence
First data file in file is as reference file.
According to the order of the key of storage in data file, data file can be ranked up.Such as, in data file 1
Comprise key1, key2, key3, data file 2 comprise key4, key5, key6, data file 3 comprises key7, key8,
According to the order of key, key9, then may determine that the order of data file is data file 1, data file 2, data file 3.
Step B, it is judged that whether reference file is last data file in the data file after sorting；If it is,
Then terminate flow process；Otherwise, it determines the memory space sum of the next data file of reference file and reference file, and perform step
Rapid C.
Step C, it is judged that whether described memory space sum exceedes default capacity-threshold；If it is, by described benchmark
The next data file of file is as reference file, and goes to step B；Otherwise, by next of reference file and reference file
Individual data Piece file mergence, the data file after merging is as reference file, and goes to step B.
Such as, data file 1 to data file n being carried out data file merging process, arranging memory capacity threshold value is 1M
(million).Using data file 1 as reference file, it is judged that the memory capacity sum of reference file and data file 2 whether more than 1M,
Assume to be not above 1M, then by 2-in-1 to data file 1 and data file and be data file 1 ', using data file 1 ' as benchmark
File, and whether continue the judgment standard file memory capacity sum with data file 3 more than 1M, it is assumed that more than 1M, then by number
According to file 3 as reference file, it is judged that whether the memory capacity sum of reference file and data file 4 is more than 1M, and the rest may be inferred,
During until reference file is last file in data file queue, terminate flow process.
The embodiment of the present invention additionally provides a kind of method of data query, and the most corresponding each data file of system sets up index
File, records the scope of the major key that each SSTable in the data file of its correspondence is stored in index file.Concrete, rope
Can record the beginning key in SSTable in quotation part, the number of key in SSTable, SSTable data are at its affiliated number
According to the skew in file.When data base starts, the content of each index file can be loaded in internal memory by system in advance.
As shown in Figure 6, the flow process of this data query method may include steps of:
Step 601, when receiving query statement, system determines the data belonging to the major key carried in described query statement
File.System can search the data literary composition corresponding for key in each more new data according to the metadata of record in meta data file
Part, concrete grammar illustrates the most, is not repeated at this.
Step 602, each according in this data file of record in index file corresponding to this data file of system
The scope of the major key that SSTable is stored, determines the SSTable belonging to the major key carried in query statement.
When data base starts, the content of each index file can be loaded in internal memory by system in advance.Then, system can
The scope of the major key to be stored according to each SSTable in the data file loaded in internal memory, determines belonging to corresponding major key
SSTable。
Step 603, searches the value that this major key is corresponding in the SSTable belonging to this major key.
In the embodiment of the present invention, receive and update instruction, and record the more new data carried in renewal instruction, more new data packets
Include major key and corresponding renewal operation content, when the first default trigger event occurs, search in each more new data of record
The data file belonging to major key, according to the described renewal operation content in more new data, by corresponding for each more new data
Data file is merged in new data file, and replaces the data file before merging by described new data file, thus, can
To improve the efficiency that non-relational database historical data updates.
Embodiment two
As it is shown in fig. 7, for the embodiment of the present invention provide data of non relational database update method in concrete applied field
Handling process in scape, may include steps of:
Step 701, system is switched to more new data merging patterns, obtains the catalogue that current data is currently in use.Assume to work as
The catalogue of front use is 0 catalogue, and the follow-up more new data received again is stored in 1 catalogue.
Step 702, obtains the key that in this catalogue, all more new datas are corresponding, and corresponding according to each key of metadata lookup
Data file.
Step 703, distribution file available numbering, set up new data file, and by number corresponding for each more new data
According in Piece file mergence to new data file.Merge after terminating, the index of the data file before merging and correspondence thereof can be deleted
File, and be released to can use by its reference number of a document.
Step 704, sets up the index file of new data file, and by the content update of this index file to internal memory.
Index file corresponding to the data file before merging corresponding contents in internal memory can be replaced by the content of this index file.
Step 705, terminates the flow process that merges of more new data and historical data, and system is switched to normal mode.At normal mode
Under formula, system only receives more new data and stores in corresponding catalogue, and is not updated merging of data and historical data.
In the embodiment of the present invention, receive and update instruction, and record the more new data carried in renewal instruction, more new data packets
Include major key and corresponding renewal operation content, when the first default trigger event occurs, search in each more new data of record
The data file belonging to major key, according to the described renewal operation content in more new data, by corresponding for each more new data
Data file is merged in new data file, and replaces the data file before merging by described new data file, thus, can
To improve the efficiency that non-relational database historical data updates.
Embodiment three
Based on identical technology design, the embodiment of the present invention additionally provides a kind of data of non relational database more new clothes
Put, as shown in Figure 8, including:
Logging modle 810, is used for receiving renewal instruction, and records the described more new data updating and carrying in instruction, described
More new data packets includes major key and corresponding renewal operation content；
Search module 820, for when the first default trigger event occurs, searching belonging to the major key in each more new data
Data file；
More new module 830, for according to the described renewal operation content in more new data, by corresponding for the most more new data
Data file be merged in new data file, and replace the data file before merging by described new data file.
Preferably, also include updating data management module, be used for:
Pre-build the catalogue of two more new datas described in record；
The more new data carried in instruction that updates received be recorded in one of them catalogue；
When described first trigger event occurs, carry out catalogue switching, will subsequently received renewal instruction be carried
More new data recorded in another catalogue, and in the catalogue that will use before record each more new data and corresponding data
Piece file mergence is in new data file；
In the catalogue that will use before, each more new data of record is merged into new data file with corresponding data file
More new data after in, in the catalogue used before deletion.
Preferably, also include merging module, be used for:
When the second default trigger event occurs, it is judged that whether the memory capacity sum of adjacent multiple data files exceedes
Preset memory capacity threshold value, without exceeding, then the plurality of data file is merged into a data file.
Preferably, described merging module, specifically for:
Step A, is ranked up data file according to the order of the major key comprised in data file, by the data after sequence
First data file in file is as reference file；
Step B, it is judged that whether reference file is last data file in the data file after sorting；If it is,
Then terminate flow process；Otherwise, it determines the memory space sum of the next data file of reference file and reference file, and perform step
Rapid C；
Step C, it is judged that whether described memory space sum exceedes default capacity-threshold；If it is, by described benchmark
The next data file of file is as reference file, and goes to step B；Otherwise, by next of reference file and reference file
Individual data Piece file mergence, the data file after merging is as reference file, and goes to step B.
Preferably, described the second default trigger event, specially reach the preset data Piece file mergence cycle.
Preferably, also include:
Set up module, set up index file for corresponding each data file, index file records the data literary composition of its correspondence
The scope of the major key that each SSTable in part is stored；
Enquiry module, for when receiving query statement, determines the number belonging to the major key carried in described query statement
According to file；The master stored according to each SSTable in this data file of record in the index file that this data file is corresponding
The scope of key, determines the SSTable belonging to the major key carried in described query statement；SSTable belonging to this major key looks into
Look for the value that this major key is corresponding.
Preferably, described the first default trigger event, specially reach to preset more new data and merge the cycle.
Preferably, also include that reference number of a document manages module, be used for:
For the reference number of a document that described new data file distribution is new；
Delete the data file being replaced, and discharge the reference number of a document of this data file.
In the embodiment of the present invention, receive and update instruction, and record the more new data carried in renewal instruction, more new data packets
Include major key and corresponding renewal operation content, when the first default trigger event occurs, search in each more new data of record
The data file belonging to major key, according to the described renewal operation content in more new data, by corresponding for each more new data
Data file is merged in new data file, and replaces the data file before merging by described new data file, thus, can
To improve the efficiency that non-relational database historical data updates.
It will be appreciated by those skilled in the art that the module in the device in embodiment can describe according to embodiment to carry out point
It is distributed in the device of embodiment, it is also possible to carry out respective change and be disposed other than in one or more devices of the present embodiment.On
The module stating embodiment can merge into a module, it is also possible to is further split into multiple submodule.
The invention described above embodiment sequence number, just to describing, does not represent the quality of embodiment.
Through the above description of the embodiments, those skilled in the art is it can be understood that can be by the present invention
Software adds the mode of required general hardware platform and realizes, naturally it is also possible to by hardware, but a lot of in the case of the former is more
Good embodiment.Based on such understanding, prior art is contributed by technical scheme the most in other words
Part can embody with the form of software product, and this computer software product is stored in a storage medium, if including
Dry instruction is with so that a station terminal equipment (can be mobile phone, personal computer, server, or the network equipment etc.) performs basis
Invent the method described in each embodiment.
The above is only the preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For Yuan, under the premise without departing from the principles of the invention, it is also possible to make some improvements and modifications, these improvements and modifications also should
Depending on protection scope of the present invention.
Claims (14)
1. a data of non relational database update method, it is characterised in that including:
Receiving and update instruction, and record and described update the more new data that carries in instruction, described more new data packets includes major key and right
The renewal operation content answered；
When the first default trigger event occurs, search the data file belonging to major key in each more new data；
According to the described renewal operation content in more new data, data file corresponding for each more new data is merged into new
In data file, and replace the data file before merging by described new data file；
Pre-build the catalogue of two more new datas described in record；
The more new data carried in instruction that updates received be recorded in one of them catalogue；
When described first trigger event occurs, carry out catalogue switching, subsequently received updating in instruction is carried more
New data records is in another catalogue, and each more new data of record and corresponding data file in the catalogue that will use before
It is merged in new data file；
In the catalogue that will use before, each more new data of record is merged in new data file it with corresponding data file
After, the more new data in the catalogue used before deletion.
2. the method for claim 1, it is characterised in that also include:
When the second default trigger event occurs, it is judged that whether the memory capacity sum of adjacent multiple data files exceedes default
Memory capacity threshold value, without exceeding, then merges into a data file by the plurality of data file.
3. method as claimed in claim 2, it is characterised in that the memory capacity sum of the adjacent multiple data files of described judgement
Whether exceed default memory capacity threshold value, without exceeding, then the plurality of data file merged into a data file,
Particularly as follows:
Step A, is ranked up data file according to the order of the major key comprised in data file, by the data file after sequence
In first data file as reference file；
Step B, it is judged that whether reference file is last data file in the data file after sorting；If it is, knot
Line journey；Otherwise, it determines the memory space sum of the next data file of reference file and reference file, and perform step C；
Step C, it is judged that whether described memory space sum exceedes default capacity-threshold；If it is, by described reference file
Next data file as reference file, and go to step B；Otherwise, by next number of reference file Yu reference file
According to Piece file mergence, the data file after merging is as reference file, and goes to step B.
4. method as claimed in claim 2, it is characterised in that described the second default trigger event, specially reaches default
Data file merges the cycle.
5. the method for claim 1, it is characterised in that also include: corresponding each data file sets up index file, index
File records the scope of the major key that each SSTable in the data file of its correspondence is stored；
When receiving query statement, determine the data file belonging to the major key carried in described query statement；
According to the major key that stored of each SSTable in this data file of record in the index file that this data file is corresponding
Scope, determines the SSTable belonging to the major key carried in described query statement；
The value that this major key is corresponding is searched in the SSTable belonging to this major key.
6. the method for claim 1, it is characterised in that described the first default trigger event, specially reaches default
More new data merges the cycle.
7. the method for claim 1, it is characterised in that also include:
For the reference number of a document that described new data file distribution is new；
Delete the data file being replaced, and discharge the reference number of a document of this data file.
8. a data of non relational database updating device, it is characterised in that including:
Logging modle, is used for receiving renewal instruction, and records the described more new data updating and carrying in instruction, described more new data
Including major key and corresponding renewal operation content；
Search module, for when the first default trigger event occurs, searching the data belonging to major key in each more new data
File；
More new module, for according to the described renewal operation content in more new data, by data corresponding for each more new data
Piece file mergence is in new data file, and replaces the data file before merging by described new data file；
Update data management module, be used for:
Pre-build the catalogue of two more new datas described in record；
The more new data carried in instruction that updates received be recorded in one of them catalogue；
When described first trigger event occurs, carry out catalogue switching, subsequently received updating in instruction is carried more
New data records is in another catalogue, and each more new data of record and corresponding data file in the catalogue that will use before
It is merged in new data file；
In the catalogue that will use before, each more new data of record is merged in new data file it with corresponding data file
After, the more new data in the catalogue used before deletion.
9. device as claimed in claim 8, it is characterised in that also include merging module, be used for:
When the second default trigger event occurs, it is judged that whether the memory capacity sum of adjacent multiple data files exceedes default
Memory capacity threshold value, without exceeding, then merges into a data file by the plurality of data file.
10. device as claimed in claim 9, it is characterised in that described merging module, specifically for:
Step A, is ranked up data file according to the order of the major key comprised in data file, by the data file after sequence
In first data file as reference file；
Step B, it is judged that whether reference file is last data file in the data file after sorting；If it is, knot
Line journey；Otherwise, it determines the memory space sum of the next data file of reference file and reference file, and perform step C；
Step C, it is judged that whether described memory space sum exceedes default capacity-threshold；If it is, by described reference file
Next data file as reference file, and go to step B；Otherwise, by next number of reference file Yu reference file
According to Piece file mergence, the data file after merging is as reference file, and goes to step B.
11. devices as claimed in claim 9, it is characterised in that described the second default trigger event, specially reach default
Data file merges the cycle.
12. devices as claimed in claim 8, it is characterised in that also include:
Set up module, set up index file for corresponding each data file, index file records in the data file of its correspondence
The scope of major key that stored of each SSTable；
Enquiry module, for when receiving query statement, determines the data literary composition belonging to the major key carried in described query statement
Part；According to the major key that stored of each SSTable in this data file of record in the index file that this data file is corresponding
Scope, determines the SSTable belonging to the major key carried in described query statement；Searching in the SSTable belonging to this major key should
The value that major key is corresponding.
13. devices as claimed in claim 8, it is characterised in that described the first default trigger event, specially reach default
More new data merges the cycle.
14. devices as claimed in claim 8, it is characterised in that also include that reference number of a document manages module, be used for:
For the reference number of a document that described new data file distribution is new；
Delete the data file being replaced, and discharge the reference number of a document of this data file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210187686.0A CN103473239B (en) | 2012-06-08 | 2012-06-08 | A kind of data of non relational database update method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210187686.0A CN103473239B (en) | 2012-06-08 | 2012-06-08 | A kind of data of non relational database update method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103473239A CN103473239A (en) | 2013-12-25 |
| CN103473239B true CN103473239B (en) | 2016-12-21 |
Family
ID=49798095
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210187686.0A Active CN103473239B (en) | 2012-06-08 | 2012-06-08 | A kind of data of non relational database update method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103473239B (en) |
Families Citing this family (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6323887B2 (en) | 2013-12-31 | 2018-05-16 | 華為技術有限公司Ｈｕａｗｅｉ Ｔｅｃｈｎｏｌｏｇｉｅｓ Ｃｏ．，Ｌｔｄ． | Method and device for changing root node |
| CN103744628B (en) * | 2014-01-27 | 2016-09-28 | 北京奇虎科技有限公司 | SSTable file storage method and device |
| CN105900093B (en) | 2014-11-12 | 2018-02-02 | 华为技术有限公司 | A kind of update method of the tables of data of KeyValue databases and table data update apparatus |
| WO2016175880A1 (en) * | 2015-04-29 | 2016-11-03 | Hewlett Packard Enterprise Development Lp | Merging incoming data in a database |
| CN111343241B (en) * | 2015-07-24 | 2022-12-09 | 创新先进技术有限公司 | Graph data updating method, device and system |
| CN106407224B (en) | 2015-07-31 | 2019-09-13 | 华为技术有限公司 | The method and apparatus of file compacting in a kind of key assignments storage system |
| CN106603610B (en) * | 2015-10-19 | 2019-11-22 | 阿里巴巴集团控股有限公司 | A kind of data acquisition and synchronous method, device |
| CN108121718B (en) * | 2016-11-28 | 2021-12-24 | 北京国双科技有限公司 | Data processing method and synchronization equipment |
| CN107016548B (en) * | 2016-12-09 | 2022-01-28 | 创新先进技术有限公司 | Client system, information processing method, reading and writing method and device |
| CN107145501B (en) * | 2017-03-20 | 2020-09-29 | 中国建设银行股份有限公司 | File updating method, file copying method and corresponding system |
| CN107315764B (en) * | 2017-05-11 | 2020-04-21 | 北京奇艺世纪科技有限公司 | Method and system for updating non-relational database associated data |
| CN107682356B (en) * | 2017-10-26 | 2020-08-04 | 广州市雷军游乐设备有限公司 | Data updating method and device, equipment and storage medium |
| CN110019212B (en) * | 2017-11-29 | 2021-06-18 | 杭州海康威视数字技术股份有限公司 | Data processing method and device and database server |
| CN110147359A (en) * | 2017-12-13 | 2019-08-20 | 北京奇虎科技有限公司 | A kind of increment generation method, device and a kind of data-updating method, device |
| CN110109866B (en) * | 2017-12-28 | 2021-11-09 | 中移（杭州）信息技术有限公司 | Method and equipment for managing file system directory |
| CN108512948B (en) * | 2018-03-21 | 2022-05-17 | 平安普惠企业管理有限公司 | Address book updating method and device, computer equipment and storage medium |
| CN109597819B (en) * | 2018-12-07 | 2020-12-18 | 北京字节跳动网络技术有限公司 | Method and apparatus for updating a database |
| CN109933573B (en) * | 2019-01-30 | 2021-06-01 | 北京融链科技有限公司 | Database service updating method, device and system |
| CN109947700B (en) * | 2019-01-30 | 2021-08-27 | 上海卫星工程研究所 | Satellite-borne imaging data management method based on multi-level file symbols |
| CN110727685B (en) * | 2019-10-09 | 2022-04-22 | 苏州浪潮智能科技有限公司 | Data compression method, equipment and storage medium based on Cassandra database |
| CN111488405B (en) * | 2020-04-16 | 2022-02-18 | 北京字节跳动网络技术有限公司 | Information updating method and device |
| CN112015819A (en) * | 2020-08-31 | 2020-12-01 | 杭州欧若数网科技有限公司 | Data updating method, device, equipment and medium for distributed graph database |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101499069A (en) * | 2008-02-02 | 2009-08-05 | 中兴通讯股份有限公司 | Internal memory database file updating method and apparatus |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100306236A1 (en) * | 2009-05-29 | 2010-12-02 | Sun Microsystems, Inc. | Data Policy Management System and Method for Managing Data |
-
2012
- 2012-06-08 CN CN201210187686.0A patent/CN103473239B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101499069A (en) * | 2008-02-02 | 2009-08-05 | 中兴通讯股份有限公司 | Internal memory database file updating method and apparatus |
Non-Patent Citations (2)
| Title |
|---|
| Nosql数据库教程之初探MongoDB-第一部;佚名;《http://m.gbin1.com/technology/javautilities/20120116gettingstartedwithmongodbpart1/index.html》;20120116;第1-10页 * |
| 键值存储【KV存储】以及SSTable介绍;佚名;《http://blog.sina.com.cn/s/blog_502d765f01010nwx.html》;20111213;第1-3页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103473239A (en) | 2013-12-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103473239B (en) | A kind of data of non relational database update method and device | |
| US10831736B2 (en) | Fast multi-tier indexing supporting dynamic update | |
| CN102521406B (en) | Distributed query method and system for complex task of querying massive structured data | |
| CN102542071B (en) | Distributed data processing system and method | |
| CN102521405B (en) | Massive structured data storage and query methods and systems supporting high-speed loading | |
| CN106066896B (en) | Application-aware big data deduplication storage system and method | |
| CN105117417B (en) | A kind of memory database Trie tree indexing means for reading optimization | |
| CN106708427A (en) | Storage method suitable for key value pair data | |
| CN105912687B (en) | Magnanimity distributed data base storage unit | |
| CN106294352B (en) | A kind of document handling method, device and file system | |
| CN103577123A (en) | Small file optimization storage method based on HDFS | |
| CN104035925B (en) | Date storage method, device and storage system | |
| CN107918612A (en) | The implementation method and device of key assignments memory system data structure | |
| CN105956123A (en) | Local updating software-based data processing method and apparatus | |
| CN103279502B (en) | A kind of framework and method with the data de-duplication file system be combined with parallel file system | |
| CN101539950A (en) | Data storage method and device | |
| CN105631003A (en) | Intelligent index establishing, inquiring and maintaining method supporting mass data classification and counting | |
| CN104239377A (en) | Platform-crossing data retrieval method and device | |
| CN106682110B (en) | Image file storage and management system and method based on Hash grid index | |
| CN106599091B (en) | RDF graph structure storage and index method based on key value storage | |
| CN104951464B (en) | Date storage method and system | |
| CN103049574B (en) | Realize key assignments file system and the method for file dynamic copies | |
| CN102779138A (en) | Hard disk access method of real time data | |
| CN103198150A (en) | Big data indexing method and system | |
| CN106155934A (en) | Based on the caching method repeating data under a kind of cloud environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20200902 Address after: 518057 Nanshan District science and technology zone, Guangdong, Zhejiang Province, science and technology in the Tencent Building on the 1st floor of the 35 layer Co-patentee after: TENCENT CLOUD COMPUTING (BEIJING) Co.,Ltd. Patentee after: TENCENT TECHNOLOGY (SHENZHEN) Co.,Ltd. Address before: Shenzhen Futian District City, Guangdong province 518057 Zhenxing Road, SEG Science Park 2 East Room 403 Patentee before: TENCENT TECHNOLOGY (SHENZHEN) Co.,Ltd. |