CN104376096A - Method for asynchronous updating based on buffer area - Google Patents
Method for asynchronous updating based on buffer area Download PDFInfo
- Publication number
- CN104376096A CN104376096A CN201410682991.6A CN201410682991A CN104376096A CN 104376096 A CN104376096 A CN 104376096A CN 201410682991 A CN201410682991 A CN 201410682991A CN 104376096 A CN104376096 A CN 104376096A
- Authority
- CN
- China
- Prior art keywords
- update request
- queue
- update
- threshold values
- renewal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for asynchronous updating based on a buffer area. The method includes the steps that an updating request is submitted; whether a queue is empty or not is judged; if the queue is not empty, queue elements are extracted, and an updating request is ready to be processed; a latest upgrading threshold value is calculated, and the updating request is updated to the local; whether upgrading triggers an upgrading strategy or not is checked; if the updating strategy is trigged, distributed catch or a database is upgraded.
Description
Technical field
The present invention relates to the renewal of data, more specifically, the present invention relates to the method for the asynchronous refresh based on buffer zone.
Background technology
If want certain value to be updated in distributed cache server or database, can only directly more new database or distributed caching.But, because code is deployed on multiple servers usually, every station server there is again multiple program example run, so the update request that the same time probably has two or more arrives distributed caching or database.
Especially for the situation upgrading certain numerical value.Fig. 1 shows in prior art the schematic diagram upgrading certain numerical value.As shown in Figure 1, such as: in distributed caching, have data A, value is 1, (each example upgrades once by the value+1 of A to have two (or two or more) program examples to wish, desirable renewal result should be 3), the value that program example synchronization obtains A from distributed caching is 1, and is 2 at program example local update, request is sent in distributed caching by synchronization, performs renewal rewards theory (being updated to 2 by 1).At this moment, although there are two program examples to have issued update request, the result upgraded is 2, instead of 3, thus be easy to produce dirty writing.Its basic reason is that two program examples all think to only have oneself in this value of renewal, because be all on the basis of 1+1, by the time expect that upgrading result is 2, and upgrades at synchronization.
Can find out, upgrade scene more frequently, produce the dirty frequency write higher (probability sending update request because of synchronization is higher).And whole acquisition, renewal rewards theory are synchronous execution, this means that each more new capital will at least mutual three ability updated value with distributed caching.In addition, in reciprocal process, the thread of execution can only be waited for, reduces the treatment effeciency of thread.
Therefore, a kind of method that can overcome the asynchronous refresh based on buffer zone of above problem is needed.
Summary of the invention
Following problem for existing disposal route: result to be updated will be waited in each more new capital, thread could continue to perform; For upgrading scene frequently, not only very large pressure is produced to distributed caching or database, and it is higher to produce the dirty probability write, the present invention proposes a kind of asynchronous, to the transparent renewal distribution formula buffer memory of user or database method, send the execution efficiency of update request thread and the lifting to database availability under the concurrent scene of height to improve.
According to one embodiment of present invention, provide a kind of method of the asynchronous refresh based on buffer zone, comprising: submit update request to; Judge whether queue is empty; If described queue is not empty, then take out queue element (QE), to prepare to process described update request; Calculate up-to-date renewal threshold value; Described update request is updated to this locality; Check whether described renewal triggers update strategy; And if trigger described update strategy, then renewal distribution formula buffer memory or database.
Preferably, the step of described submission update request comprises further: the value being gone out hash code by the cipher key calculation in described update request; By the number delivery by described hash code and processing threads, calculate side-play amount, to find corresponding processing threads; And by putting into the processing queue of corresponding processing threads after described update request encapsulation, obtain for thread.
Preferably, the step putting into the processing queue of corresponding processing threads after described update request encapsulation is comprised further: the described update request of same type is pointed to same processing threads, and puts into same processing queue.
Preferably, if described queue is empty, then current thread is sunk into sleep, and recovers to perform after waiting the thread of update request to be committed to put into update request.
Preferably, the described step calculating up-to-date renewal threshold value comprises further: setting protection threshold values before each update request calculates, obtain the size of current thread processing queue, if a) size of current thread processing queue is less than protection threshold values, then the renewal threshold values of current update request is set to described protection threshold values; If b) size of current thread processing queue is greater than protection threshold values, then directly adopt the size of current queue as current renewal threshold values.
Preferably, Trigger update strategy comprise update times reach described up-to-date renewal threshold values or this upgrade within the update cycle.
The present invention judges to ask by the mode of which thread computes, achieves each update request and to be fixed thread process, thus when thread process upgrades, without any need for locking, have good execution efficiency.And the present invention proposes a kind of mode of asynchronous refresh to update request, improve while upgrading efficiency, serve the effect of buffer zone, have protective effect when a large amount of request reaches suddenly to distributed caching or database.
According to the detailed description below the disclosure and accompanying drawing, other object, feature and advantage will be apparent to those skilled in the art.
Accompanying drawing explanation
Accompanying drawing illustrates embodiments of the invention, and is used from instructions one and explains principle of the present invention.In the accompanying drawings:
Fig. 1 shows in prior art the schematic diagram upgrading certain numerical value.
Fig. 2 is according to an embodiment of the invention for submitting the schematic diagram of the thread of update request to.
Fig. 3 is the internal storage structure schematic diagram upgrading buffer zone according to an embodiment of the invention.
Fig. 4 is according to an embodiment of the invention based on the process flow diagram of the method for the asynchronous refresh of buffer zone.
Embodiment
A kind of method of the asynchronous refresh based on buffer zone is disclosed according to embodiments of the invention.In the following description, for illustrative purposes, multiple detail has been set forth to provide the complete understanding to embodiments of the invention.But it is evident that for those skilled in the art, embodiments of the invention can realize when not having these details.
" thread pool " is a kind of multiple threads form as used in this, adds task to queue in processing procedure, then after establishment thread, automatically starts these tasks.Thread pool threads is all background thread.Each thread uses the storehouse size of acquiescence, runs, and be in multiple thread units with the priority given tacit consent to.In Java, the normal thread pool that uses is executed the task under multi-thread environment, for the treatment of the read-write of multiple file, and the functions such as analysis.
" lock " is at the conventional tool-class of the multi-thread environment next one as used in this, and for doing synchronous operation between code block, ensure in the scope of lock covering, the serial of code performs, and is usually used in solving concurrent problem.
" thread " refers to the thread in Java as used in this, and it is the minimum unit of run time version.Usually, under the unified management of thread pool, each category code is performed.
" distributed memory system " refers to and data scatter is stored in multiple stage independently on equipment as used herein.Traditional network store system adopts the storage server concentrated to deposit all data, and storage server becomes the bottleneck of system performance, is also the focus of reliability and security, can not meet the needs of Mass storage application.Distributed network storage system adopts extendible system architecture, utilizes multiple stage storage server to share storage load, utilizes location server to locate storage information, and it not only increases the reliability of system, availability and access efficiency, is also easy to expansion.
According to the technical solution adopted for the present invention to solve the technical problems, result cache only will be upgraded in program local when accepting update request, reach the threshold values of user's setting when update times or in distributed caching or database, send update request again after periodic certain time period of interval, to realize the buffering of update request in program local, reduce the service pressure of distributed caching or database, reduce the request amount sent to distributed caching or database simultaneously.Meanwhile, when this update strategy of use renewal threshold values, according to the frequent degree upgraded, dynamic adjustment upgrades threshold values, to realize when tackling a large amount of update request arrived out of the blue, play the effect of request in program local buffering, to reduce the pressure to distributed caching or database.
It should be noted that, the invention provides a kind of extraordinary solution and provide some acquiescence implementations, and there is good extendability, except the executive mode of acquiescence, the interface additionally providing trigger policy and persistence strategy realizes " when Trigger update " voluntarily for user, " be updated in distributed caching or database in ", has very strong dirigibility.
Fig. 2 is according to an embodiment of the invention for submitting the schematic diagram of the thread of update request to.As shown in Figure 2, for the thread submitting update request to, specific implementation step is as follows:
1. when there being update request to produce, calling more new interface of the present invention, submitting update request to;
2. the present invention is by the key (key) in update request, calculates the value of hash code (hashcode), and gets by the number of hashcode and processing threads and touch, and calculates side-play amount, thus finds corresponding processing threads;
3. be put in the processing queue of processing threads after update request being encapsulated, obtain for thread.
The numerical value of the hashcode int type that to be jdk calculate according to the address of object or character string or numeral.The implication of delivery (Mod) computing is complementation.Such as 11Mod 2, value is 1, and any number and 2 deliverys, result may be only 0 or 1, and the result namely remmed can only for being less than by remainder to the integer be more than or equal between 0.
According to embodiments of the invention, certain thread is made only to process the update request of some by the calculating of hashcode and processing threads number delivery, and because the key value of each update request is constant, hashcode would not become, remainder also can not become, thus the update request of this key has fixing thread process at every turn, such benefit is update times statistics, the synchronous execution that lock is not needed during this buffer memory initialization, high and give, Appropriate application multithreading dispersion treatment update request, and because do not need the relation locked, improve treatment effeciency.
Fig. 3 is the internal storage structure schematic diagram upgrading buffer zone according to an embodiment of the invention.As shown in Figure 3, thread and request queue one_to_one corresponding.The feature of Fig. 3 content structure is update request to be assigned in different request queues by hash algorithm of the present invention, uses queue and has accomplished the asynchronous refresh (as long as request is submitted to request queue by update request just complete renewal rewards theory) of asking.And normally direct database or the distributed caching that update request is sent to of existing technology, after waiting for that database or distributed caching are updated successfully, just calculate and complete renewal rewards theory.The most obvious shortcoming of existing way is: 1. will to wait for that database or distributed caching are updated successfully just complete for update request, upgrades more consuming time; 2. each update request arrives and all needs to be updated to database or distributed caching, has larger request pressure to database or distributed caching, and the present invention is because be upgrade after asynchronous merging update request, so above-mentioned two shortcomings all do not exist.
Fig. 4 is according to an embodiment of the invention based on the process flow diagram of the method for the asynchronous refresh of buffer zone.Particularly, for processing threads: specific implementation step is as follows:
1. judge whether queue is empty, and if it is empty, then current thread is sunk into sleep, recover to perform after waiting the thread of update request to be committed to put into update request.If not empty, then second step is carried out;
2. take out queue element (QE), prepare process update request;
3. calculate up-to-date renewal threshold values;
4. update request is updated to this locality;
5. check whether this renewal triggers update strategy; If meet update condition, then perform the 6th step, otherwise, turn back to the first step, continue to obtain update request process;
6. renewal result up-to-date for this locality is updated in distributed caching or database.
The situation of Trigger update strategy includes but not limited to, update times reaches the up-to-date renewal threshold values of step 3 setting, or this upgraded within the update cycle, specifically determines according to the strategy of specifying when using.The present invention provide not only update times or update cycle as the update condition triggering distributed caching or database update, and provides the interface of trigger condition and update mode, makes user customize trigger condition and update mode more flexibly.But; although it will be understood by those skilled in the art that and the invention provides update times or update cycle as trigger condition, the present invention includes but be not limited to this two kinds of judgment modes; in principle, any method that can judge whether that needs upgrade all should in protection scope of the present invention.
The present invention is by the algorithm of dynamic calculation threshold values: during each process update request; the protection threshold values of the length of processing queue with artificial setting is compared; get the renewal threshold values of the greater as request; achieve associating of renewal frequency and update request number; make the present invention when tackling a large amount of update request; the renewal threshold values of request can increase along with request amount, reduces renewal frequency, makes there is better protective effect to distributed caching or database.And after request amount reduces, threshold values reduces gradually, and renewal frequency increases gradually.Ensure that while improving feasibility and upgrade fast being as far as possible updated in distributed caching or database of result.Achieve the balance of availability and promptness.
Such as, according to an embodiment, if the update strategy used is the strategy upgraded by update times, then use following algorithm to calculate up-to-date renewal threshold values, so that dynamic conditioning upgrades threshold values, this algorithm hypothesis renewal distribution formula buffer memory or database consuming time local consuming time higher than renewal far away, arthmetic statement is as follows: the size obtaining current thread processing queue, if a) size of current thread processing queue is less than the minimum threshold values of system default, then the threshold values of current update request is set to this minimum threshold values; If b) size of current thread processing queue is greater than minimum threshold values, then directly adopt the size of current queue as the threshold values of current update request.
This is because: when asking to arrive in a large number, or renewal distribution formula buffer memory or database slow, when causing the processing queue of thread constantly to increase, the corresponding rising of threshold values, thus reduce the frequency of renewal distribution formula buffer memory or database, accelerate processing locality speed, thus reduce the size of processing queue.When queue shortens, illustrate that the submission of update request is not very frequent, therefore, upgrade threshold values and shorten together with queue length, the frequency of renewal distribution formula buffer memory or database raises, and what guarantee update request can be fast as far as possible is updated in distributed caching or database.
By technical scheme of the present invention, can realize the asynchronization upgraded, more new thread need not wait result to be updated can continue to perform; By upgrading threshold values or the reasonable set of update cycle, the dirty generation writing situation can be avoided.
And, pass through the present invention, greatly can reduce the renewal pressure to distributed caching or database persistence service, particularly when update requests a large amount of suddenly, because there is the setting dynamically updating threshold values, thus avoid to the full extent shining into distributed caching or database asking pressure accordingly, improve the availability of distributed caching or database.
Above-described embodiment is only the preferred embodiments of the present invention, is not limited to the present invention.It will be apparent for a person skilled in the art that without departing from the spirit and scope of the present invention, various amendment and change can be carried out to embodiments of the invention.Therefore, the invention is intended to contain all amendments within the scope of the present invention as defined by the appended claims of falling into or modification.
Claims (6)
1., based on a method for the asynchronous refresh of buffer zone, comprising:
Submit update request to;
Judge whether queue is empty;
If described queue is not empty, then take out queue element (QE), to prepare to process described update request;
Calculate up-to-date renewal threshold value;
Described update request is updated to this locality;
Check whether described renewal triggers update strategy; And
If trigger described update strategy, then renewal distribution formula buffer memory or database.
2. method according to claim 1, wherein, the step of described submission update request comprises further:
The value of hash code is gone out by the cipher key calculation in described update request;
By the number delivery by described hash code and processing threads, calculate side-play amount, to find corresponding processing threads; And
By putting into the processing queue of corresponding processing threads after described update request encapsulation, obtain for thread.
3. method according to claim 2, wherein the step putting into the processing queue of corresponding processing threads after described update request encapsulation is comprised further: the described update request of same type is pointed to same processing threads, and puts into same processing queue.
4. method according to claim 1, comprises further: if described queue is for empty, then current thread is sunk into sleep, and recovers to perform after waiting the thread of update request to be committed to put into update request.
5. method according to claim 1 and 2, wherein, described in calculate up-to-date renewal threshold value step comprise further:
Setting protection threshold values before each update request calculates,
Obtain the size of current thread processing queue,
If a) size of current thread processing queue is less than protection threshold values, then the renewal threshold values of current update request is set to described protection threshold values;
If b) size of current thread processing queue is greater than protection threshold values, then directly adopt the size of current queue as current renewal threshold values.
6. method according to claim 1, wherein, Trigger update strategy comprise update times reach described up-to-date renewal threshold values or this upgrade within the update cycle.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410682991.6A CN104376096B (en) | 2014-11-24 | 2014-11-24 | The method of asynchronous refresh based on buffering area |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410682991.6A CN104376096B (en) | 2014-11-24 | 2014-11-24 | The method of asynchronous refresh based on buffering area |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104376096A true CN104376096A (en) | 2015-02-25 |
| CN104376096B CN104376096B (en) | 2018-07-13 |
Family
ID=52555003
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410682991.6A Active CN104376096B (en) | 2014-11-24 | 2014-11-24 | The method of asynchronous refresh based on buffering area |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104376096B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105930397A (en) * | 2016-04-15 | 2016-09-07 | 北京思特奇信息技术股份有限公司 | Message processing method and system |
| CN107464151A (en) * | 2016-06-02 | 2017-12-12 | 北京京东尚科信息技术有限公司 | The order data processing method and processing device of high concurrent business |
| CN109597818A (en) * | 2018-11-28 | 2019-04-09 | 优刻得科技股份有限公司 | Data-updating method, device, storage medium and equipment |
| CN109725840A (en) * | 2017-10-30 | 2019-05-07 | 伊姆西Ip控股有限责任公司 | It is throttled using asynchronous wash away to write-in |
| CN111061654A (en) * | 2019-11-11 | 2020-04-24 | 支付宝(杭州)信息技术有限公司 | Cache refreshing processing method and device and electronic equipment |
| CN111241134A (en) * | 2019-12-31 | 2020-06-05 | 五八有限公司 | Data processing method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1901548A (en) * | 2006-07-26 | 2007-01-24 | 白杰 | Buffer zone managing method |
| US20080313238A1 (en) * | 2004-10-27 | 2008-12-18 | International Business Machines Corporation | Read-Copy Update System And Method |
| CN103514177A (en) * | 2012-06-20 | 2014-01-15 | 盛趣信息技术（上海）有限公司 | Data storage method and system |
| CN104102634A (en) * | 2013-04-02 | 2014-10-15 | 北京本果信息技术有限公司 | Method and system for synchronizing database |
-
2014
- 2014-11-24 CN CN201410682991.6A patent/CN104376096B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080313238A1 (en) * | 2004-10-27 | 2008-12-18 | International Business Machines Corporation | Read-Copy Update System And Method |
| CN1901548A (en) * | 2006-07-26 | 2007-01-24 | 白杰 | Buffer zone managing method |
| CN103514177A (en) * | 2012-06-20 | 2014-01-15 | 盛趣信息技术（上海）有限公司 | Data storage method and system |
| CN104102634A (en) * | 2013-04-02 | 2014-10-15 | 北京本果信息技术有限公司 | Method and system for synchronizing database |
Non-Patent Citations (1)
| Title |
|---|
| 陈珉: ""分布式空间数据库主动数据更新研究"", 《中国优秀博硕士学位论文全文数据库（博士）基础科学辑（季刊）》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105930397A (en) * | 2016-04-15 | 2016-09-07 | 北京思特奇信息技术股份有限公司 | Message processing method and system |
| CN105930397B (en) * | 2016-04-15 | 2019-05-17 | 北京思特奇信息技术股份有限公司 | A kind of message treatment method and system |
| CN107464151A (en) * | 2016-06-02 | 2017-12-12 | 北京京东尚科信息技术有限公司 | The order data processing method and processing device of high concurrent business |
| CN109725840A (en) * | 2017-10-30 | 2019-05-07 | 伊姆西Ip控股有限责任公司 | It is throttled using asynchronous wash away to write-in |
| CN109725840B (en) * | 2017-10-30 | 2022-04-05 | 伊姆西Ip控股有限责任公司 | Throttling writes with asynchronous flushing |
| CN109597818A (en) * | 2018-11-28 | 2019-04-09 | 优刻得科技股份有限公司 | Data-updating method, device, storage medium and equipment |
| CN111061654A (en) * | 2019-11-11 | 2020-04-24 | 支付宝(杭州)信息技术有限公司 | Cache refreshing processing method and device and electronic equipment |
| CN111061654B (en) * | 2019-11-11 | 2022-05-10 | 支付宝(杭州)信息技术有限公司 | Cache refreshing processing method and device and electronic equipment |
| CN111241134A (en) * | 2019-12-31 | 2020-06-05 | 五八有限公司 | Data processing method and device |
| CN111241134B (en) * | 2019-12-31 | 2023-08-08 | 五八有限公司 | Data processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104376096B (en) | 2018-07-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10681175B2 (en) | Multi-server node service processing and consensus method and device | |
| CN104376096A (en) | Method for asynchronous updating based on buffer area | |
| US9928264B2 (en) | High performance transactions in database management systems | |
| US9292448B2 (en) | Dynamic sizing of memory caches | |
| US8352517B2 (en) | Infrastructure for spilling pages to a persistent store | |
| US9195599B2 (en) | Multi-level aggregation techniques for memory hierarchies | |
| US9348752B1 (en) | Cached data replication for cache recovery | |
| US9632852B2 (en) | Detecting and managing flooding of multi-tenant message queues | |
| US10534776B2 (en) | Proximity grids for an in-memory data grid | |
| CN107958018B (en) | Method and device for updating data in cache and computer readable medium | |
| US9229869B1 (en) | Multi-lock caches | |
| CN111078147A (en) | Processing method, device and equipment for cache data and storage medium | |
| CN102880557A (en) | Multistage distribution type high-speed cache of heterogeneous data source | |
| CN105187327A (en) | Distributed message queue middleware | |
| US11663143B2 (en) | Multi-state midtier dynamic cache replacement | |
| CN106095835A (en) | Client side picture four-layer caching method | |
| CN105069152A (en) | Data processing method and apparatus | |
| US8341368B2 (en) | Automatic reallocation of structured external storage structures | |
| US20140136791A1 (en) | Managing data within a cache | |
| US9027026B2 (en) | Managing transactions within an application server | |
| Wei et al. | A GPU-specialized inference parameter server for large-scale deep recommendation models | |
| US11269784B1 (en) | System and methods for efficient caching in a distributed environment | |
| US9921759B2 (en) | Multithreaded memory manager to de-allocate memory objects using private freelists | |
| CN116594752A (en) | Flow scheduling method, device, equipment, medium and program product | |
| CN105630556B (en) | A kind of method and system loading resource |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |