CN102867070A - Method for updating cache of key-value distributed memory system - Google Patents
Method for updating cache of key-value distributed memory system Download PDFInfo
- Publication number
- CN102867070A CN102867070A CN2012103780400A CN201210378040A CN102867070A CN 102867070 A CN102867070 A CN 102867070A CN 2012103780400 A CN2012103780400 A CN 2012103780400A CN 201210378040 A CN201210378040 A CN 201210378040A CN 102867070 A CN102867070 A CN 102867070A
- Authority
- CN
- China
- Prior art keywords
- key
- value
- key word
- buffer memory
- cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention discloses a method for updating cache of a key-value distributed memory system. The method includes: initiating inquiry operation, distributing a column name for each list, and building a keyword; inquiring database and storing a result S if data of the keyword corresponding to cache do not exist, and directly returning to the result S if corresponding cache exists; inputting the returned result S in a key-value system by the inquiry operation using the keyword key, and enabling a corresponding value of the keyword to be S; building a keyword group by the inquiry operation according to the column names; registering the keyword in the corresponding value of the keyword group; returning to the result S; initiating updating operation and building the keyword group according to updated conditions; inquiring whether the corresponding value exists in the cache according to the keyword group, and deleting the corresponding value and completing updating if the corresponding value exists; and otherwise, stopping updating. By the method for updating cache of the key-value distributed memory system, the cache of the key-value distributed memory system can be kept persistently, and the related cache can be eliminated immediately during data updating, and a user can be guaranteed to see the latest data.
Description
Technical field
The present invention relates to computer realm, particularly relate to a kind of method of the key-value of renewal distributed memory system buffer memory.
Background technology
Along with the develop rapidly of internet, use the people of network also more and more, the request that how to respond fast the user has been that each Internet firm strives for one of most important means of user.For example the content that provides of website is fine, very valuable, but the each access of user all will wait for for tens seconds, and the time has been grown the user and is certain to run off so.The most crucial technology point that solves fast response is exactly how to utilize efficiently buffer memory, because with respect to from the relation data library inquiry, it is fast many that the access data of directly postponing is wanted, usually from buffer memory, fetch data minimum faster more than 10 times than from relational database, fetching data, and can support larger Concurrency Access amount.
Relational database: hereinafter to be referred as database, be the database that is based upon on the relational model of database basis, come data in the process database by means of concept and methodologies such as algebras of sets.The relational database of main flow has oracle, SQL, access, db2, sqlserver, sybase etc. at present.A relational database is to comprise the one group of form that enters within the predefined kind.Each form (sometimes being called as a relation) comprises the one or more data class that shows with tabulation.Every row comprises a unique data entity, and these data are listed as the kind of definition.For instance, typical commercial order articles database can comprise the form of a customer information of description of showing with tabulation: name, address, telephone number, etc.A form in addition can be described an order: product, client, date, selling price, etc.A user of database can obtain to be fit to the viewpoint of database of this user's needs.For instance, office manager of branch office may like having bought all clients' of product viewpoint or report after a specific date.The finance service of same company manager may obtain from identical form the account's that paid about needs report.The advantage of relational database is exactly powerful clearly, can be according to various conditional filterings, shortcoming also clearly, when access surpassed 1000 simultaneously, it is very low that performance just becomes.
Buffer memory: the buffer memory of saying here refers to the distributed storage scheme based on key-value, key-value distributed memory system inquiry velocity is fast, the store data amount is large, it is high concurrent to support, be fit to very much inquire about by major key, but can not carry out complicated condition query.If be aided with Real-Time Search Engine(real-time search engine) carry out complex conditions retrieval, full-text search, just can substitute the lower relevant databases such as MySQL of concurrency performance, reach high concurrent, high-performance, save the purpose of tens times of number of servers.Key-value distributed storage take MemcacheDB, Tokyo Tyrant as representative under concurrent connections up to ten thousand, is finished at a high speed inquiry like a cork.And MySQL under a hundreds of concurrent connection, has just collapsed basically.
Internet firm recognizes that substantially all relational database has become the website and done the strongest large bottleneck now, generally also all can use buffer memory to preserve the result who inquires from database, to satisfy larger concurrency.But two problems can be arranged all generally, first, can not support very large visit capacity at certain table when upgrading, but when table has renewal, owing to can not delete accurately relevant buffer memory, the buffer memory of whole table all can only be cleared out, although next time, data cached and database was consistent like this, but the moment that buffer memory empties, the pressure of database becomes greatly immediately, and system collapses possibly.Second, for the previous case, with the time that buffer memory keeps shorter, when Data Update is arranged, need not remove the buffer memory of whole table, wait that it is naturally expired, such benefit is that the pressure of data can not increase suddenly, but shortcoming also is that apparent, new exactly data can not be by real-time the seeing of user.
Summary of the invention
The technical problem to be solved in the present invention provides a kind of method of the key-value of renewal distributed memory system buffer memory, the buffer memory of being correlated with in Data Update when buffer memory can be kept lastingly can be eliminated immediately, can guarantee that the data that the user sees are up-to-date.
For solving the problems of the technologies described above, the present invention upgrades the method for key-value distributed memory system buffer memory, comprising:
(1) initiate query manipulation, for each table distributes a row name, this key word represents this inquiry, is used for checking the data whether buffer memory is arranged from the key-value system;
The data that do not have the corresponding buffer memory of described key word, Query Database saving result S, if there is corresponding buffer memory, so direct return results S;
(2) utilize query manipulation to use described key word key, the as a result S that returns is put in the key-value system, value value corresponding to key word is S in the recording step (1);
(3) query manipulation makes up key word group keygroup according to described row name, is used for preserving the key word key of step (1);
(4) key word is registered among value value corresponding to key word group;
(5) return results S;
(6) initiate to upgrade operation, more new database;
(7) upgrade operation according to update condition adopt with step (3) in identical algorithm structure key word group;
(8) inquire about according to this key group keygroup whether corresponding value value is arranged in buffer memory, if exist corresponding value then in the key-value system, to delete respective value, finish renewal; If do not exist corresponding value to finish, without upgrading.
Wherein, implementation step (1) adopts algorithm " prefix _ table name _ SQL condition " to make up key word key.
Wherein, implementation step (3) adopts algorithm " table name _ row name _ value " to make up key word group keygroup.
The present invention incites somebody to action simple key word key=by the one query request on traditional buffering scheme basis〉value value structure, be configured to key word group keyGroup=〉key word key=〉value value structure, when having data that renewal is arranged, can calculate once relevant key word of the inquiry key according to key word group keyGroup by update request, and delete these buffer memorys, realize the renewal of buffer memory.
Description of drawings
The present invention is further detailed explanation below in conjunction with accompanying drawing and embodiment:
Fig. 1 is process flow diagram of the present invention.
Embodiment
As shown in Figure 1, the present invention upgrades the method for key-value distributed memory system buffer memory, comprising:
(1) initiate query manipulation, for each table distributes a row name, adopt simultaneously algorithm " prefix _ table name _ SQL condition " to make up key word key, this key word represents this inquiry, is used for checking the data whether buffer memory is arranged from the key-value system;
The data that do not have the corresponding buffer memory of described key word, Query Database saving result S, if there is corresponding buffer memory, so direct return results S;
(2) utilize query manipulation to use described key word key, the as a result S that returns is put in the key-value system, value value corresponding to key word is S in the recording step (1);
(3) query manipulation adopts algorithm " table name _ row name _ value " to make up key word group keygroup according to described row name, is used for preserving the key word key of step (1);
(4) key word is registered among the value value of key word group;
(5) return results S;
(6) initiate to upgrade operation, more new database;
(7) upgrade operation according to update condition adopt with step (3) in identical algorithm structure key word group;
(8) inquire about according to this key group keygroup whether corresponding value value is arranged in buffer memory, if exist corresponding value then in the key-value system, to delete respective value, finish renewal; If do not exist corresponding value to finish, without upgrading.
During implementation, at first state an object, comprise query manipulation find, deletion action delete upgrades operation update, update insert method;
Call the query manipulation find of this object, initiate query manipulation, for each table distributes a row name, make up simultaneously key word key(and adopt algorithm " prefix _ table name _ SQL condition "), this key word key represents this inquiry, is used for checking the data whether buffer memory is arranged from the key-value system;
Do not have the data of the corresponding buffer memory of described key word, Query Database obtains as a result S; If there is corresponding buffer memory, so direct return results S;
Utilize query manipulation find to use described key word key, the as a result S that returns is put in the key-value system, value value corresponding to key word is S in the recording step (1), forms to concern key=〉S
Query manipulation find makes up key word group keygroup(according to described row name and adopts algorithm " table name _ row name _ value "), be used for preserving the key word key of step (1);
Query manipulation find registers to key word key to form among the value value of correspondence of key word group keygroup and concerns keygroup=〉key=〉S
Return results S;
Initiate to upgrade operating update, upgrade more new database of operation update, upgrade operation update and construct a key word group keyGroup(employing algorithm " table name _ row name _ value " according to update condition);
Upgrade operation update and inquire corresponding value value according to keyGroup in the key-value system cache, if exist corresponding value then to delete respective value in the key-value system, finish renewal, this moment, this value value was key; If do not exist corresponding value to finish, without upgrading.
The key that key uses query manipulation find to produce before being, might this inquiry also can be called, so upgrade value corresponding to operation update deletion key this moment, next like this, if use query manipulation find still to call the key query caching, because former buffer memory is deleted, so query manipulation find can go to search in the database up-to-date content automatically.
Below through the specific embodiment and the embodiment the present invention is had been described in detail, but these are not to be construed as limiting the invention.In the situation that do not break away from the principle of the invention, those skilled in the art also can make many distortion and improvement, and these also should be considered as protection scope of the present invention.
Claims (3)
1. a method of upgrading key-value distributed memory system buffer memory is characterized in that, comprising:
(1) initiate query manipulation, for each table distributes a row name, make up key word key, this key word represents this inquiry; The data that do not have the corresponding buffer memory of described key word, Query Database saving result S, if there is corresponding buffer memory, so direct return results S;
(2) utilize query manipulation to use described key word key, the as a result S that returns is put in the key-value system, value value corresponding to key word is S in the recording step (1);
(3) query manipulation makes up key word group keygroup according to described row name, is used for preserving the key word key of step (1);
(4) key word is registered among value value corresponding to key word group;
(5) return results S;
(6) initiate to upgrade operation, more new database;
(7) upgrade operation according to update condition adopt with step (3) in identical algorithm structure key word group;
(8) inquire about according to this key group keygroup whether corresponding value value is arranged in buffer memory, if exist corresponding value then in the key-value system, to delete respective value, finish renewal; If do not exist corresponding value to finish, without upgrading.
2. upgrade as claimed in claim 1 the method for key-value distributed memory system buffer memory, it is characterized in that: implementation step (1) adopts algorithm " prefix _ table name _ SQL condition " to make up key word key.
3. upgrade as claimed in claim 1 the method for key-value distributed memory system buffer memory, it is characterized in that: implementation step (3) adopts algorithm " table name _ row name _ value " to make up key word group keygroup.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012103780400A CN102867070A (en) | 2012-09-29 | 2012-09-29 | Method for updating cache of key-value distributed memory system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012103780400A CN102867070A (en) | 2012-09-29 | 2012-09-29 | Method for updating cache of key-value distributed memory system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102867070A true CN102867070A (en) | 2013-01-09 |
Family
ID=47445939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012103780400A Pending CN102867070A (en) | 2012-09-29 | 2012-09-29 | Method for updating cache of key-value distributed memory system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102867070A (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103218455A (en) * | 2013-05-07 | 2013-07-24 | 中国人民解放军国防科学技术大学 | Method of high-speed concurrent processing of user requests of Key-Value database |
| CN103338243A (en) * | 2013-06-20 | 2013-10-02 | 新浪网技术(中国)有限公司 | Method and system for updating cache data of Web node |
| CN103870393A (en) * | 2013-07-09 | 2014-06-18 | 携程计算机技术（上海）有限公司 | Cache management method and system |
| CN103902698A (en) * | 2014-03-31 | 2014-07-02 | 北京车商汇软件有限公司 | Data storage system and data storage method |
| CN103902701A (en) * | 2014-03-31 | 2014-07-02 | 北京车商汇软件有限公司 | Data storage system and data storage method |
| CN103902702A (en) * | 2014-03-31 | 2014-07-02 | 北京车商汇软件有限公司 | Data storage system and data storage method |
| CN104050276A (en) * | 2014-06-26 | 2014-09-17 | 北京思特奇信息技术股份有限公司 | Cache processing method and system of distributed database |
| CN104102669A (en) * | 2013-04-12 | 2014-10-15 | 同程网络科技股份有限公司 | Method for checking air ticket |
| CN104765782A (en) * | 2015-03-20 | 2015-07-08 | 五八同城信息技术有限公司 | Index sequencing updating method and device |
| CN105516346A (en) * | 2015-12-30 | 2016-04-20 | 广东欧珀移动通信有限公司 | Method and device for analyzing cache strategy |
| CN105740383A (en) * | 2016-01-27 | 2016-07-06 | 中国科学院计算技术研究所 | Method and system for realizing data consistency of database and cache in big data platform |
| CN105988899A (en) * | 2015-03-05 | 2016-10-05 | 广州市动景计算机科技有限公司 | Method and device for realizing data caching |
| CN106649585A (en) * | 2016-11-18 | 2017-05-10 | 福建中金在线信息科技有限公司 | Retrieval method and device |
| CN106919628A (en) * | 2015-12-28 | 2017-07-04 | 阿里巴巴集团控股有限公司 | A kind for the treatment of method and apparatus of diagram data |
| CN107508878A (en) * | 2017-08-17 | 2017-12-22 | 杭州时趣信息技术有限公司 | A kind of key name delet method and device based on caching proxy server |
| CN108153911A (en) * | 2018-01-24 | 2018-06-12 | 广西师范学院 | The distributed cloud storage method of data |
| CN108255959A (en) * | 2017-12-21 | 2018-07-06 | 武汉斑马快跑科技有限公司 | Data query method and device in a kind of Redis |
| CN109491987A (en) * | 2018-11-01 | 2019-03-19 | 恒生电子股份有限公司 | Data managing method and device |
| CN111046106A (en) * | 2019-12-19 | 2020-04-21 | 杭州中恒电气股份有限公司 | Cache data synchronization method, device, equipment and medium |
| CN112559244A (en) * | 2019-09-25 | 2021-03-26 | 北京国双科技有限公司 | Data processing method and device, electronic equipment and computer readable medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117338A (en) * | 2011-04-02 | 2011-07-06 | 天脉聚源(北京)传媒科技有限公司 | Data base caching method |
| US20110246518A1 (en) * | 2010-04-01 | 2011-10-06 | Verizon Patent And Licensing Inc. | Method and system of distributed caching |
-
2012
- 2012-09-29 CN CN2012103780400A patent/CN102867070A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110246518A1 (en) * | 2010-04-01 | 2011-10-06 | Verizon Patent And Licensing Inc. | Method and system of distributed caching |
| CN102117338A (en) * | 2011-04-02 | 2011-07-06 | 天脉聚源(北京)传媒科技有限公司 | Data base caching method |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104102669A (en) * | 2013-04-12 | 2014-10-15 | 同程网络科技股份有限公司 | Method for checking air ticket |
| CN103218455A (en) * | 2013-05-07 | 2013-07-24 | 中国人民解放军国防科学技术大学 | Method of high-speed concurrent processing of user requests of Key-Value database |
| CN103338243A (en) * | 2013-06-20 | 2013-10-02 | 新浪网技术(中国)有限公司 | Method and system for updating cache data of Web node |
| CN103870393A (en) * | 2013-07-09 | 2014-06-18 | 携程计算机技术（上海）有限公司 | Cache management method and system |
| CN103902701A (en) * | 2014-03-31 | 2014-07-02 | 北京车商汇软件有限公司 | Data storage system and data storage method |
| CN103902702A (en) * | 2014-03-31 | 2014-07-02 | 北京车商汇软件有限公司 | Data storage system and data storage method |
| CN103902698B (en) * | 2014-03-31 | 2018-04-13 | 北京皮尔布莱尼软件有限公司 | A kind of data-storage system and storage method |
| CN103902702B (en) * | 2014-03-31 | 2017-11-28 | 北京皮尔布莱尼软件有限公司 | A kind of data-storage system and storage method |
| CN103902698A (en) * | 2014-03-31 | 2014-07-02 | 北京车商汇软件有限公司 | Data storage system and data storage method |
| CN104050276A (en) * | 2014-06-26 | 2014-09-17 | 北京思特奇信息技术股份有限公司 | Cache processing method and system of distributed database |
| CN104050276B (en) * | 2014-06-26 | 2017-08-01 | 北京思特奇信息技术股份有限公司 | The method for caching and processing and system of a kind of distributed data base |
| CN105988899A (en) * | 2015-03-05 | 2016-10-05 | 广州市动景计算机科技有限公司 | Method and device for realizing data caching |
| CN105988899B (en) * | 2015-03-05 | 2019-04-02 | 广州市动景计算机科技有限公司 | The method and apparatus for realizing data buffer storage |
| CN104765782A (en) * | 2015-03-20 | 2015-07-08 | 五八同城信息技术有限公司 | Index sequencing updating method and device |
| CN104765782B (en) * | 2015-03-20 | 2019-06-21 | 五八同城信息技术有限公司 | A kind of index order update method and device |
| CN106919628A (en) * | 2015-12-28 | 2017-07-04 | 阿里巴巴集团控股有限公司 | A kind for the treatment of method and apparatus of diagram data |
| CN105516346A (en) * | 2015-12-30 | 2016-04-20 | 广东欧珀移动通信有限公司 | Method and device for analyzing cache strategy |
| CN105516346B (en) * | 2015-12-30 | 2018-09-21 | 广东欧珀移动通信有限公司 | A kind of analysis method and device of cache policy |
| CN105740383A (en) * | 2016-01-27 | 2016-07-06 | 中国科学院计算技术研究所 | Method and system for realizing data consistency of database and cache in big data platform |
| CN106649585A (en) * | 2016-11-18 | 2017-05-10 | 福建中金在线信息科技有限公司 | Retrieval method and device |
| CN107508878A (en) * | 2017-08-17 | 2017-12-22 | 杭州时趣信息技术有限公司 | A kind of key name delet method and device based on caching proxy server |
| CN107508878B (en) * | 2017-08-17 | 2020-08-11 | 杭州时趣信息技术有限公司 | Key name deleting method and device based on cache proxy server |
| CN108255959A (en) * | 2017-12-21 | 2018-07-06 | 武汉斑马快跑科技有限公司 | Data query method and device in a kind of Redis |
| CN108153911A (en) * | 2018-01-24 | 2018-06-12 | 广西师范学院 | The distributed cloud storage method of data |
| CN109491987A (en) * | 2018-11-01 | 2019-03-19 | 恒生电子股份有限公司 | Data managing method and device |
| CN112559244A (en) * | 2019-09-25 | 2021-03-26 | 北京国双科技有限公司 | Data processing method and device, electronic equipment and computer readable medium |
| CN111046106A (en) * | 2019-12-19 | 2020-04-21 | 杭州中恒电气股份有限公司 | Cache data synchronization method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102867070A (en) | Method for updating cache of key-value distributed memory system | |
| CN104850572B (en) | HBase non-primary key index construct and querying method and its system | |
| CN104123340B (en) | A kind of database divides table paging query method and system | |
| EP2605158B1 (en) | Mixed join of row and column database tables in native orientation | |
| EP3519986B1 (en) | Direct table association in in-memory databases | |
| US20120310934A1 (en) | Historic View on Column Tables Using a History Table | |
| US9183267B2 (en) | Linked databases | |
| US10025710B2 (en) | Pattern for integrating primary and secondary data stores in a sharded data domain | |
| US20160253382A1 (en) | System and method for improving a query response rate by managing a column-based store in a row-based database | |
| US11119742B2 (en) | Cache efficient reading of result values in a column store database | |
| JP2003006036A (en) | Clustered application server and web system having database structure | |
| US10437688B2 (en) | Enhancing consistent read performance for in-memory databases | |
| EP3814938A1 (en) | Techniques for enabling and integrating in-memory semi-structered data and text document searches with in-memory columnar query processing | |
| KR101400214B1 (en) | Appratus for providing a hybrid c interface | |
| US9229960B2 (en) | Database management delete efficiency | |
| CN102339315A (en) | Index updating method and system of advertisement data | |
| WO2011130706A2 (en) | Methods and systems for performing cross store joins in a multi-tenant store | |
| CN102968456B (en) | A kind of raster data reading and processing method and device | |
| CN105740383A (en) | Method and system for realizing data consistency of database and cache in big data platform | |
| CN108665335B (en) | Method for processing shopping cart data of shopping website | |
| EP3182296A1 (en) | Dbms-supported score assignment | |
| CN101404649B (en) | Data processing system based on CACHE and its method | |
| CN103365987B (en) | Clustered database system and data processing method based on shared-disk framework | |
| JP5235483B2 (en) | A method and apparatus for maintaining consistency between a database and a virtual table. | |
| US9229969B2 (en) | Management of searches in a database system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20130109 |