CN105447167A - Processing method and apparatus for node cache data in distributed system - Google Patents
Processing method and apparatus for node cache data in distributed system Download PDFInfo
- Publication number
- CN105447167A CN105447167A CN201510883615.8A CN201510883615A CN105447167A CN 105447167 A CN105447167 A CN 105447167A CN 201510883615 A CN201510883615 A CN 201510883615A CN 105447167 A CN105447167 A CN 105447167A
- Authority
- CN
- China
- Prior art keywords
- data key
- data
- node
- buffer memory
- cached
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Abstract
The invention discloses a processing method and apparatus for node cache data in a distributed system. The method comprises: reading at least one cache datum of a node, wherein each cache datum is in the form of a data key value pair; for each cache datum, extracting a corresponding data key and performing persistent storage on the data key; and when the node is restarted, loading the data key into a cache of the node. According to the scheme, the time for performing the persistent storage on the cache data is greatly shortened, the occupation of system disk resources is reduced, and the loading speed is increased.
Description
Technical field
The present invention relates to Internet technical field, be specifically related to the data cached disposal route of a kind of distributed system interior joint and device.
Background technology
Buffer memory is the buffer zone of exchanges data, when system needs certain data of reading database, first can search these data from buffer memory, if found these data in the buffer, then directly perform, if do not find these data in the buffer, then search from database again.Because the operating speed by buffer memory is searched faster than from database, therefore buffer memory can help system run rapidly, reduces the time delay of system.
But when node is restarted, if buffer memory is not through persistent storage, buffer memory will be lost, and the loss of buffer memory can cause system travelling speed within a period of time comparatively slow, brings bad experience sense to user.Therefore, in the prior art in order to avoid cache miss, be generally carry out persistent storage by data cached, but when data cached data volume is larger, data cached persistent storage not only by the time a large amount of for cost, also can take a large amount of system disk resources.
Summary of the invention
In view of the above problems, propose the present invention to provide a kind of overcoming the problems referred to above or the data cached disposal route of the distributed system interior joint that solves the problem at least in part and device, greatly reduce the data cached time of carrying out needed for persistent storage, decrease taking system disk resource, improve loading velocity.
According to an aspect of the present invention, provide the disposal route that a kind of distributed system interior joint is data cached, the method comprises:
Read at least one data cached of node, data cached have the right form of data key values;
Data cached for each, extract corresponding data key, and data key is carried out persistent storage;
When node is restarted, data key is loaded in the buffer memory of node.
According to a further aspect in the invention, provide the treating apparatus that a kind of distributed system interior joint is data cached, this device comprises:
Read module, be suitable for reading at least one data cached of node, data cached have the right form of data key values;
Extraction module, is suitable for for each data cached, extracts corresponding data key;
Memory module, the data key being suitable for extraction module to extract carries out persistent storage;
First load-on module, is suitable for, when node is restarted, being loaded in the buffer memory of node by data key.
According to technical scheme provided by the invention, read at least one data cached of node, wherein, data cached have the right form of data key values, then data cached for each, extract corresponding data key, and data key is carried out persistent storage, finally when node is restarted, data key is loaded in the buffer memory of node.Technical scheme provided by the invention has only carried out persistent storage to the data key in data cached, thus greatly reduce the data cached time of carrying out needed for persistent storage, decrease taking system disk resource, in addition, according to technical scheme provided by the invention, when node is restarted, only data key is loaded in the buffer memory of node, improve loading velocity, optimize the processing mode that distributed system interior joint is data cached.
Above-mentioned explanation is only the general introduction of technical solution of the present invention, in order to technological means of the present invention can be better understood, and can be implemented according to the content of instructions, and can become apparent, below especially exemplified by the specific embodiment of the present invention to allow above and other objects of the present invention, feature and advantage.
Accompanying drawing explanation
By reading hereafter detailed description of the preferred embodiment, various other advantage and benefit will become cheer and bright for those of ordinary skill in the art.Accompanying drawing only for illustrating the object of preferred implementation, and does not think limitation of the present invention.And in whole accompanying drawing, represent identical parts by identical reference symbol.In the accompanying drawings:
Fig. 1 shows the schematic flow sheet of the disposal route that distributed system interior joint is data cached according to an embodiment of the invention;
Fig. 2 shows the schematic flow sheet of the disposal route that distributed system interior joint is data cached in accordance with another embodiment of the present invention;
Fig. 3 shows the illustrative view of functional configuration of the treating apparatus that distributed system interior joint is data cached according to an embodiment of the invention;
Fig. 4 shows the illustrative view of functional configuration of the treating apparatus that distributed system interior joint is data cached in accordance with another embodiment of the present invention.
Embodiment
Below with reference to accompanying drawings exemplary embodiment of the present disclosure is described in more detail.Although show exemplary embodiment of the present disclosure in accompanying drawing, however should be appreciated that can realize the disclosure in a variety of manners and not should limit by the embodiment set forth here.On the contrary, provide these embodiments to be in order to more thoroughly the disclosure can be understood, and complete for the scope of the present disclosure can be conveyed to those skilled in the art.
Fig. 1 shows the schematic flow sheet of the disposal route that distributed system interior joint is data cached according to an embodiment of the invention, and as shown in Figure 1, the method comprises the steps:
Step S100, read at least one data cached of node, data cached have the right form of data key values.
In the step s 100, read at least one data cached of node, wherein, data cached have the right form of data key values, i.e. the data cached form with key-value, wherein, key is data key, value is data value, and data value can be the data of various ways, and the form right by data key values can manage data cached easily.
Step S101, data cached for each, extract corresponding data key, and data key is carried out persistent storage.
Step S100 have read node at least one data cached after, perform step S101, data cached for each, extract corresponding data key, and data key is carried out persistent storage, that is, in step S101, only extract the data key of data cached middle correspondence, do not extract corresponding data value, and only data key is carried out persistent storage.Due in most of the cases, data key length is general shorter, and the length of data value is long, if data key and data value are all carried out persistent storage, not only by the time a large amount of for cost, but also a large amount of system disk resources can be taken, compared with therefore all carrying out persistent storage with data key with data value, only persistent storage is carried out to data key and greatly reduce the time needed for persistent storage of carrying out and taking system disk resource.
Step S102, when node is restarted, is loaded into data key in the buffer memory of node.
When node is restarted, perform step S102, the data key through persistent storage is loaded in the buffer memory of node, owing to only need be loaded in the buffer memory of node by data key, without the need to being loaded in the buffer memory of node by data value simultaneously, thus improves loading velocity.
The disposal route that the distributed system interior joint provided according to the present embodiment is data cached, read at least one data cached of node, wherein, data cached have the right form of data key values, then data cached for each, extract corresponding data key, and data key is carried out persistent storage, finally when node is restarted, data key is loaded in the buffer memory of node.Technical scheme provided by the invention has only carried out persistent storage to the data key in data cached, thus greatly reduce the data cached time of carrying out needed for persistent storage, decrease taking system disk resource, in addition, according to technical scheme provided by the invention, when node is restarted, only data key is loaded in the buffer memory of node, improve loading velocity, optimize the processing mode that distributed system interior joint is data cached.
Fig. 2 shows the schematic flow sheet of the disposal route that distributed system interior joint is data cached in accordance with another embodiment of the present invention, and as shown in Figure 2, the method comprises the steps:
Step S200, read at least one data cached of node, data cached have the right form of data key values.
Wherein, data cached have the right form of data key values, and the form right by data key values can manage data cached easily.
Step S201, data cached for each, extract the expired time stamp of corresponding data key and data key, and for data key setting is for identifying the zone bit without data value, data key and zone bit are combined as data key values pair.
Owing to only extracting the data key of data cached middle correspondence in step s 201, do not extract corresponding data value, therefore, the data value that data key is corresponding is empty, for the ease of distinguishing these data keies, in step s 201 also for data key is arranged for identifying the zone bit without data value, then data key and zone bit are combined as data key values pair.
In addition, due to expired time can be arranged to data cached, therefore, the expired time stamp of data key is also extracted in step s 201.
Step S202, by data key values to and data key expired time stamp carry out persistent storage.
After step S201 is complete, step S202 by data key values to and data key expired time stamp carry out persistent storage.Because data key values is combined into by data key and zone bit, therefore by the data key values be combined into by data key and zone bit to carrying out persistent storage, compared with data key all carries out persistent storage with data value, reduce the time of carrying out needed for persistent storage.In addition, for the ease of in subsequent step, the expired time according to data key is stabbed, and judges whether data key is expired, also needs to carry out persistent storage to the expired time stamp of data key in step S202.
Step S203, when node is restarted, by data key values to being loaded in the buffer memory of node.
When node is restarted, the data key values be combined into by data key and zone bit to being loaded in the buffer memory of node, due to without the need to being loaded in the buffer memory of node by data value, thus is improve loading velocity by step S203.
Step S204, when node is restarted, the expired time according to data key is stabbed, and judges that whether data key is out of date; If so, then step S205 is performed; If not, then step S206 is performed.
When node is restarted, step S204 stabs according to the expired time of data key, judges that whether data key is out of date; If judge, data key is out of date, then perform step S205, delete this data key; If judge, data key is not out of date, then perform step S206.
Step S205, deletes data key.
If judge that data key is out of date through step S204, then delete data key out of date.
Step S206, receives the updating message that the data key values of other node transmission is right.
After node is restarted, if there is other node to send the right updating message of data key values, so perform step S206, receive the updating message that the data key values of other node transmission is right.
Step S207, according to updating message, by data key values corresponding in the buffer memory of node to the data key values pair being updated to the transmission of other node.
In step S207, according to the updating message that step S206 receives, by data key values corresponding in the buffer memory of node to the data key values pair being updated to the transmission of other node.
Such as, in step S206, have received other node send about data key values to 1 updating message, and according to updating message, data key values is updated to data value 1 to the data value in 1, so in step S207, data key values corresponding in the buffer memory of node is updated to the data key values of other node transmission to 1 to 1.Suppose, in the buffer memory of node, corresponding data key values was not updated 1, so this data key values has zone bit in 1, so in step S207, data key values corresponding in the buffer memory of node is updated to data key values that other node sends to the data value 1 in 1 to the zone bit in 1.
In addition, the method also can comprise: if receive the data cached request of access for node, then search data value corresponding to the data key that stores in the buffer memory of other node according to accessed data key, data value is loaded in the buffer memory of node, and responds the step of request of access return data value.Particularly, realize to step S210 by step S208.
Step S208, if the data cached request of access receiving for node, the data key values of accessing loaded in the buffer memory of decision node is to whether comprising zone bit; If so, then step S209 is performed; If not, then step S210 is performed.
Due to when receiving the data cached request of access for node, the data key values of accessing loaded in the buffer memory of node is to may not be updated, therefore, in step S208, the data key values of accessing that loads in the buffer memory to node is needed to judge whether comprising zone bit.If judge that accessed data key values is to comprising zone bit, then perform step S209; If judge that accessed data key values is not to comprising zone bit, then perform step S210.
Step S209, searches data value corresponding to the data key that stores in the buffer memory of other node according to accessed data key, and the zone bit arranged for data key is replaced with data value.
If judge that accessed data key values is to comprising zone bit through step S208, then represent the data key values centering of accessing without data value, then need to perform step S209, data value corresponding to the data key that stores in the buffer memory of other node is searched according to accessed data key, the zone bit arranged for data key is replaced with data value, thus makes this data key values centering have corresponding data value.After step S209 is complete, then perform step S210.
Step S210, the data value that response request of access return data key-value pair comprises.
If judge that accessed data key values is not to comprising zone bit through step S208, then represent the data key values centering of accessing and there is corresponding data value, then directly can perform step S210, the data value that response request of access return data key-value pair comprises.Or after the zone bit arranged for data key is replaced with data value by step S209, make data key values centering have corresponding data value, then then step S210, response request of access returns this data value.
The disposal route that the distributed system interior joint provided according to the present embodiment is data cached, data cached for each, not only extract corresponding data key, also extract the expired time stamp of data key, when restarting with convenient node, expired time according to data key is stabbed, and deletes data key out of date; For the ease of distinguishing, for data key is arranged for identifying the zone bit without data value, data key and zone bit are combined as data key values pair, then when node is restarted, by data key values to being loaded in the buffer memory of node, and the updating message that the data key values sent according to other node is right, by data key values corresponding in the buffer memory of node to the data key values pair being updated to other node and sending.
In addition, according to technical scheme provided by the invention, after receiving the data cached request of access for node, only judging that accessed data key values is to when comprising zone bit, just search data value corresponding to the data key that stores in the buffer memory of other node according to accessed data key, the zone bit arranged for data key is replaced with data value, and responds request of access return data value; Judging that accessed data key values is not to when comprising zone bit, then directly respond the data value that request of access return data key-value pair comprises.Not only ensure that when node is restarted according to technical scheme provided by the invention, data cachedly not lose, also greatly reduce the data cached time of carrying out needed for persistent storage, decrease taking system disk resource, improve the loading velocity data cached when node is restarted, and only judging that accessed data key values is to when comprising zone bit, just need to search data value corresponding to the data key that stores in the buffer memory of other node according to accessed data key, thus improve response speed, optimize the processing mode that distributed system interior joint is data cached further.
Fig. 3 shows the illustrative view of functional configuration of the treating apparatus that distributed system interior joint is data cached according to an embodiment of the invention, as shown in Figure 3, this device comprises: read module 301, extraction module 302, memory module 303 and the first load-on module 304.
Read module 301 is suitable for: read at least one data cached of node, data cached have the right form of data key values.
Read module 301 reads at least one data cached of node, wherein, data cached have the right form of data key values, the i.e. data cached form with key-value, wherein, key is data key, and value is data value, and data value can be the data of various ways, the form right by data key values can manage data cached easily.
Extraction module 302 is suitable for: data cached for each, extracts corresponding data key.
Read module 301 have read node at least one data cached after, trigger extraction module 302, data cached for each, extract corresponding data key, that is, only extract the data key of data cached middle correspondence at extraction module 302, do not extract corresponding data value.
Memory module 303 is suitable for: the data key extracted by extraction module 301 carries out persistent storage.
Because extraction module 302 only extracts the data key of data cached middle correspondence, do not extract corresponding data value, and in most of the cases, data key length is general shorter, and the length of data value is long, if data key and data value are all carried out persistent storage, not only by the time a large amount of for cost, but also a large amount of system disk resources can be taken, therefore compared with all carrying out persistent storage with data key with data value, memory module 303 only carries out persistent storage to data key, greatly reduce the time of carrying out needed for persistent storage, and decrease taking of system disk resource.
First load-on module 304 is suitable for: when node is restarted, be loaded in the buffer memory of node by data key.
When node is restarted, trigger the first load-on module 304, data key through memory module 303 persistent storage is loaded in the buffer memory of node, because data key only need be loaded in the buffer memory of node by the first load-on module 304, without the need to being loaded in the buffer memory of node by data value simultaneously, thus improve loading velocity.
The treating apparatus that the distributed system interior joint provided according to the present embodiment is data cached, read module reads at least one data cached of node, wherein, data cached have the right form of data key values, then extraction module is data cached for each, and extract corresponding data key, data key is carried out persistent storage by memory module, finally when node is restarted, data key is loaded in the buffer memory of node by the first load-on module.Technical scheme provided by the invention has only carried out persistent storage to the data key in data cached, thus greatly reduce the data cached time of carrying out needed for persistent storage, decrease taking system disk resource, in addition, according to technical scheme provided by the invention, when node is restarted, only data key is loaded in the buffer memory of node, improve loading velocity, optimize the processing mode that distributed system interior joint is data cached.
Fig. 4 shows the illustrative view of functional configuration of the treating apparatus that distributed system interior joint is data cached in accordance with another embodiment of the present invention, as shown in Figure 4, this device comprises: read module 401, extraction module 402, memory module 403, first load-on module 404, access modules 405, second load-on module 406 and respond module 407.
Read module 401 is suitable for: read at least one data cached of node, data cached have the right form of data key values.
Extraction module 402 is suitable for: data cached for each, extracts corresponding data key.
Read module 401 have read node at least one data cached after, trigger extraction module 402, data cached for each, extract corresponding data key, do not extract corresponding data value.
Memory module 403 is suitable for: data key is carried out persistent storage.
Data cached for each at extraction module 402, after extracting corresponding data key, trigger memory module 403, extracted data key is carried out persistent storage.
First load-on module 404 is suitable for: when node is restarted, be loaded in the buffer memory of node by data key.
When node is restarted, trigger the first load-on module 404, the data key of memory module 403 persistent storage is loaded in the buffer memory of node.
Access modules 405 is suitable for: receive the data cached request of access for node.
Second load-on module 406 is suitable for: if access modules 405 receives the data cached request of access for node, then search data value corresponding to the data key that stores in the buffer memory of other node according to accessed data key, data value is loaded in the buffer memory of node.
Respond module 407 is suitable for: response request of access return data value.
After data value to be loaded in the buffer memory of node by the second load-on module 406, trigger respond module 407, response request of access return data value.
Because extraction module 402 only extracts the data key of data cached middle correspondence, do not extract corresponding data value, therefore, the data value that data key is corresponding is empty, for the ease of distinguishing these data keies, this device also comprises: arrange module 408, is suitable for data key and arranges for identifying the zone bit without data value, data key and zone bit are combined as data key values pair.
When arranging for identifying the zone bit without data value for data key, memory module 403 is further adapted for: by data key values to carrying out persistent storage; First load-on module 404 is further adapted for: when node is restarted, by data key values to being loaded in the buffer memory of node.
After node is restarted, likely other node can send the right updating message of data key values, so this device also can comprise further: update module 409, be suitable for receiving the updating message that the data key values of other node transmission is right, according to updating message, by data key values corresponding in the buffer memory of node to the data key values pair being updated to the transmission of other node.
Due to when receiving the data cached request of access for node, the data key values of accessing loaded in the buffer memory of node is to may not be updated, therefore this device also comprises: the first judge module 410, if be suitable for access modules 405 to receive data cached request of access for node, the data key values of accessing loaded in the buffer memory of decision node is to whether comprising zone bit.
Now, second load-on module 406 is further adapted for: if the first judge module 410 judges that accessed data key values is to comprising zone bit, search data value corresponding to the data key that stores in the buffer memory of other node according to accessed data key, and the zone bit arranged for data key is replaced with data value.After the zone bit arranged for data key is replaced with data value by the second load-on module 406, trigger respond module 407, response request of access return data value.
Respond module 407 is further adapted for: if the first judge module 410 judges that accessed data key values is to not comprising zone bit, the data value that response request of access return data key-value pair comprises.
In addition, owing to can arrange expired time to data cached, therefore, the extraction module 402 in this device is further adapted for: the expired time stamp extracting data key from data cached; Memory module 403 is further adapted for: the expired time of data key stamp is carried out persistent storage.
In order to the expired time stamp according to data key judges whether data key is expired, and process data key out of date, this device also comprises: the second judge module 411 and removing module 412.
Wherein, the second judge module 411 is suitable for: when node is restarted, and the expired time according to data key is stabbed, and judges that whether data key is out of date.If the second judge module 411 judges that data key is out of date, then trigger removing module 412, delete data key.
The treating apparatus that the distributed system interior joint provided according to the present embodiment is data cached, data cached for each, extraction module not only extracts corresponding data key, also extract the expired time stamp of data key, when restarting with convenient node, second judge module stabs according to the expired time of data key, judges that whether data key is out of date; For the ease of distinguishing, arranging module is that data key is arranged for identifying the zone bit without data value, data key and zone bit are combined as data key values pair, then when node is restarted, first load-on module by data key values to being loaded in the buffer memory of node, and the updating message that update module is right according to the data key values that other node sends, by data key values corresponding in the buffer memory of node to the data key values pair being updated to the transmission of other node.
In addition, according to technical scheme provided by the invention, after the data cached request of access that access modules receives for node, only judge that accessed data key values is to when comprising zone bit at the first judge module, just search data value corresponding to the data key that stores in the buffer memory of other node according to accessed data key, the zone bit arranged for data key is replaced with data value, and responds request of access return data value; Judge that accessed data key values is not to when comprising zone bit, then directly trigger respond module at the first judge module.Not only ensure that when node is restarted according to technical scheme provided by the invention, data cachedly not lose, also greatly reduce the data cached time of carrying out needed for persistent storage, decrease taking system disk resource, improve the loading velocity data cached when node is restarted, and only judging that accessed data key values is to when comprising zone bit, just need to search data value corresponding to the data key that stores in the buffer memory of other node according to accessed data key, thus improve response speed, optimize the processing mode that distributed system interior joint is data cached further.
Intrinsic not relevant to any certain computer, virtual system or miscellaneous equipment with display at this algorithm provided.Various general-purpose system also can with use based on together with this teaching.According to description above, the structure constructed required by this type systematic is apparent.In addition, the present invention is not also for any certain programmed language.It should be understood that and various programming language can be utilized to realize content of the present invention described here, and the description done language-specific is above to disclose preferred forms of the present invention.
In instructions provided herein, describe a large amount of detail.But can understand, embodiments of the invention can be put into practice when not having these details.In some instances, be not shown specifically known method, structure and technology, so that not fuzzy understanding of this description.
Similarly, be to be understood that, in order to simplify the disclosure and to help to understand in each inventive aspect one or more, in the description above to exemplary embodiment of the present invention, each feature of the present invention is grouped together in single embodiment, figure or the description to it sometimes.But, the method for the disclosure should be construed to the following intention of reflection: namely the present invention for required protection requires feature more more than the feature clearly recorded in each claim.Or rather, as claims below reflect, all features of disclosed single embodiment before inventive aspect is to be less than.Therefore, the claims following embodiment are incorporated to this embodiment thus clearly, and wherein each claim itself is as independent embodiment of the present invention.
Those skilled in the art are appreciated that and adaptively can change the module in the equipment in embodiment and they are arranged in one or more equipment different from this embodiment.Module in embodiment or unit or assembly can be combined into a module or unit or assembly, and multiple submodule or subelement or sub-component can be put them in addition.Except at least some in such feature and/or process or unit be mutually repel except, any combination can be adopted to combine all processes of all features disclosed in this instructions (comprising adjoint claim, summary and accompanying drawing) and so disclosed any method or equipment or unit.Unless expressly stated otherwise, each feature disclosed in this instructions (comprising adjoint claim, summary and accompanying drawing) can by providing identical, alternative features that is equivalent or similar object replaces.
In addition, those skilled in the art can understand, although embodiments more described herein to comprise in other embodiment some included feature instead of further feature, the combination of the feature of different embodiment means and to be within scope of the present invention and to form different embodiments.Such as, in the following claims, the one of any of embodiment required for protection can use with arbitrary array mode.
All parts embodiment of the present invention with hardware implementing, or can realize with the software module run on one or more processor, or realizes with their combination.It will be understood by those of skill in the art that the some or all functions that microprocessor or digital signal processor (DSP) can be used in practice to realize according to the some or all parts in the embodiment of the present invention.The present invention can also be embodied as part or all equipment for performing method as described herein or device program (such as, computer program and computer program).Realizing program of the present invention and can store on a computer-readable medium like this, or the form of one or more signal can be had.Such signal can be downloaded from internet website and obtain, or provides on carrier signal, or provides with any other form.
The present invention will be described instead of limit the invention to it should be noted above-described embodiment, and those skilled in the art can design alternative embodiment when not departing from the scope of claims.In the claims, any reference symbol between bracket should be configured to limitations on claims.Word " comprises " not to be got rid of existence and does not arrange element in the claims or step.Word "a" or "an" before being positioned at element is not got rid of and be there is multiple such element.The present invention can by means of including the hardware of some different elements and realizing by means of the computing machine of suitably programming.In the unit claim listing some devices, several in these devices can be carry out imbody by same hardware branch.Word first, second and third-class use do not represent any order.Can be title by these word explanations.
The invention discloses:
The disposal route that A1, a kind of distributed system interior joint are data cached, described method comprises:
Read at least one data cached of node, described data cachedly have the right form of data key values;
Data cached for each, extract corresponding data key, and described data key is carried out persistent storage;
When described node is restarted, described data key is loaded in the buffer memory of node.
A2, method according to A1, wherein, after described node is restarted and described data key is loaded into the buffer memory of node, described method also comprises:
If receive the data cached request of access for node, the data value that the described data key that stores in the buffer memory of other node is corresponding is then searched according to accessed data key, described data value is loaded in the buffer memory of node, and responds described request of access and return described data value.
A3, method according to A1 or A2, wherein, data cached for each, extract corresponding data key, and the step of described data key being carried out persistent storage comprises further:
For described data key is arranged for identifying the zone bit without data value, described data key and zone bit are combined as data key values pair;
By data key values to carrying out persistent storage.
A4, method according to A3, wherein, when described node is restarted, comprise the step that described data key is loaded in the buffer memory of node further:
When described node is restarted, by described data key values to being loaded in the buffer memory of node.
A5, method according to A3 or A4, wherein, after described node is restarted and described data key is loaded into the buffer memory of node, described method also comprises:
If receive the data cached request of access for node, judge that the data key values of accessing loaded in the buffer memory of described node is to whether comprising described zone bit;
If judge, accessed data key values is to comprising described zone bit, then search in the buffer memory of other node the data value that the described data key that stores is corresponding according to accessed data key, and the zone bit arranged for described data key is replaced with described data value;
If judge, accessed data key values is not to comprising described zone bit, then respond described request of access and return described data key values to the data value comprised.
A6, method according to any one of A1-A5, wherein, data cached for each, extract corresponding data key, and the step of described data key being carried out persistent storage comprises further:
From described data cached extract the expired time stamp of data key and described data key, and the expired time of described data key and described data key stamp is carried out persistent storage.
A7, method according to A6, wherein, when described node is restarted, described method also comprises: the expired time according to described data key is stabbed, and judges that whether described data key is out of date; If so, described data key is then deleted;
If judge the data cached request of access that described data key is not out of date and receive for node, then search in the buffer memory of other node the data value that the described data key that stores is corresponding according to accessed data key.
A8, method according to any one of A1-A7, wherein, described method also comprises:
Receive the updating message that the data key values of other node transmission is right;
According to described updating message, by data key values corresponding in the buffer memory of described node to the data key values pair being updated to described other node transmission.
The treating apparatus that B9, a kind of distributed system interior joint are data cached, described device comprises:
Read module, is suitable for reading at least one data cached of node, and described data cachedly have the right form of data key values;
Extraction module, is suitable for for each data cached, extracts corresponding data key;
Memory module, the described data key being suitable for described extraction module to extract carries out persistent storage;
First load-on module, is suitable for, when described node is restarted, being loaded in the buffer memory of node by described data key.
B10, device according to B9, wherein, described device also comprises:
Access modules, is suitable for receiving the data cached request of access for node;
Second load-on module, if be suitable for described access modules to receive data cached request of access for node, then search the data value that the described data key that stores in the buffer memory of other node is corresponding according to accessed data key, described data value is loaded in the buffer memory of node;
Respond module, is suitable for responding described request of access and returns described data value.
B11, device according to B10, wherein, described device also comprises: arrange module, is suitable for described data key and arranges for identifying the zone bit without data value, described data key and zone bit are combined as data key values pair;
Described memory module is further adapted for: by data key values to carrying out persistent storage.
B12, device according to B11, wherein, described first load-on module is further adapted for: when described node is restarted, by described data key values to being loaded in the buffer memory of node.
B13, device according to B11 or B12, wherein, described device also comprises: the first judge module, if be suitable for described access modules to receive data cached request of access for node, judges that the data key values of accessing loaded in the buffer memory of described node is to whether comprising described zone bit;
Described second load-on module is further adapted for: if described first judge module judges that accessed data key values is to comprising described zone bit, search in the buffer memory of other node the data value that the described data key that stores is corresponding according to accessed data key, and the zone bit arranged for described data key is replaced with described data value;
Described respond module is further adapted for: if described first judge module judging that accessed data key values is not to comprising described zone bit, responding described request of access and returning described data key values to the data value comprised.
B14, device according to any one of B10-B13, wherein, described extraction module is further adapted for: from described data cached extract the expired time stamp of data key and described data key;
Described memory module is further adapted for: the expired time stamp of described data key and described data key is carried out persistent storage.
B15, device according to B14, wherein, described device comprises: the second judge module and removing module;
Described second judge module, is suitable for when described node is restarted, and the expired time according to described data key is stabbed, and judges that whether described data key is out of date; If described second judge module judges that described data key is out of date, then trigger described removing module;
Described removing module, is suitable for deleting described data key.
B16, device according to any one of B9-B15, wherein, described device also comprises: update module, be suitable for receiving the updating message that the data key values of other node transmission is right, according to described updating message, by data key values corresponding in the buffer memory of described node to the data key values pair being updated to described other node transmission.
Claims (10)
1. the disposal route that distributed system interior joint is data cached, described method comprises:
Read at least one data cached of node, described data cachedly have the right form of data key values;
Data cached for each, extract corresponding data key, and described data key is carried out persistent storage;
When described node is restarted, described data key is loaded in the buffer memory of node.
2. method according to claim 1, wherein, after described node is restarted and described data key is loaded into the buffer memory of node, described method also comprises:
If receive the data cached request of access for node, the data value that the described data key that stores in the buffer memory of other node is corresponding is then searched according to accessed data key, described data value is loaded in the buffer memory of node, and responds described request of access and return described data value.
3. method according to claim 1 and 2, wherein, data cached for each, extract corresponding data key, and the step of described data key being carried out persistent storage comprises further:
For described data key is arranged for identifying the zone bit without data value, described data key and zone bit are combined as data key values pair;
By data key values to carrying out persistent storage.
4. method according to claim 3, wherein, when described node is restarted, comprises the step that described data key is loaded in the buffer memory of node further:
When described node is restarted, by described data key values to being loaded in the buffer memory of node.
5. the method according to claim 3 or 4, wherein, after described node is restarted and described data key is loaded into the buffer memory of node, described method also comprises:
If receive the data cached request of access for node, judge that the data key values of accessing loaded in the buffer memory of described node is to whether comprising described zone bit;
If judge, accessed data key values is to comprising described zone bit, then search in the buffer memory of other node the data value that the described data key that stores is corresponding according to accessed data key, and the zone bit arranged for described data key is replaced with described data value;
If judge, accessed data key values is not to comprising described zone bit, then respond described request of access and return described data key values to the data value comprised.
6. the method according to any one of claim 1-5, wherein, data cached for each, extract corresponding data key, and the step of described data key being carried out persistent storage comprises further:
From described data cached extract the expired time stamp of data key and described data key, and the expired time of described data key and described data key stamp is carried out persistent storage.
7. method according to claim 6, wherein, when described node is restarted, described method also comprises: the expired time according to described data key is stabbed, and judges that whether described data key is out of date; If so, described data key is then deleted;
If judge the data cached request of access that described data key is not out of date and receive for node, then search in the buffer memory of other node the data value that the described data key that stores is corresponding according to accessed data key.
8. the method according to any one of claim 1-7, wherein, described method also comprises:
Receive the updating message that the data key values of other node transmission is right;
According to described updating message, by data key values corresponding in the buffer memory of described node to the data key values pair being updated to described other node transmission.
9. the treating apparatus that distributed system interior joint is data cached, described device comprises:
Read module, is suitable for reading at least one data cached of node, and described data cachedly have the right form of data key values;
Extraction module, is suitable for for each data cached, extracts corresponding data key;
Memory module, the described data key being suitable for described extraction module to extract carries out persistent storage;
First load-on module, is suitable for, when described node is restarted, being loaded in the buffer memory of node by described data key.
10. device according to claim 9, wherein, described device also comprises:
Access modules, is suitable for receiving the data cached request of access for node;
Second load-on module, if be suitable for described access modules to receive data cached request of access for node, then search the data value that the described data key that stores in the buffer memory of other node is corresponding according to accessed data key, described data value is loaded in the buffer memory of node;
Respond module, is suitable for responding described request of access and returns described data value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510883615.8A CN105447167B (en) | 2015-12-04 | 2015-12-04 | The data cached processing method and processing device of distributed system interior joint |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510883615.8A CN105447167B (en) | 2015-12-04 | 2015-12-04 | The data cached processing method and processing device of distributed system interior joint |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105447167A true CN105447167A (en) | 2016-03-30 |
| CN105447167B CN105447167B (en) | 2019-03-01 |
Family
ID=55557343
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510883615.8A Active CN105447167B (en) | 2015-12-04 | 2015-12-04 | The data cached processing method and processing device of distributed system interior joint |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105447167B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106021468A (en) * | 2016-05-17 | 2016-10-12 | 上海携程商务有限公司 | Updating method and system for distributed caches and local caches |
| CN106790562A (en) * | 2016-12-26 | 2017-05-31 | 中国建设银行股份有限公司 | A kind of distributed cache system, method and device |
| CN107844386A (en) * | 2016-09-19 | 2018-03-27 | 北京金山云网络技术有限公司 | A kind of data backup, restoration methods and device |
| CN108628817A (en) * | 2017-03-15 | 2018-10-09 | 腾讯科技（深圳）有限公司 | A kind of data processing method and device |
| CN110334092A (en) * | 2019-06-14 | 2019-10-15 | 福建天泉教育科技有限公司 | Distributed ID generation method, storage medium |
| CN111177196A (en) * | 2019-12-25 | 2020-05-19 | 海尔优家智能科技（北京）有限公司 | Time window aggregation calculation method and device |
| CN113626377A (en) * | 2020-05-06 | 2021-11-09 | 杭州海康微影传感科技有限公司 | Bare data storage control method, device and equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110082908A1 (en) * | 2009-10-06 | 2011-04-07 | Bela Ban | Dynamic caching of nodes |
| CN102117338A (en) * | 2011-04-02 | 2011-07-06 | 天脉聚源(北京)传媒科技有限公司 | Data base caching method |
| CN102902730A (en) * | 2012-09-10 | 2013-01-30 | 新浪网技术(中国)有限公司 | Method and device for reading data based on data cache |
| CN103870393A (en) * | 2013-07-09 | 2014-06-18 | 携程计算机技术（上海）有限公司 | Cache management method and system |
| CN104572920A (en) * | 2014-12-27 | 2015-04-29 | 北京奇虎科技有限公司 | Data arrangement method and data arrangement device |
-
2015
- 2015-12-04 CN CN201510883615.8A patent/CN105447167B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110082908A1 (en) * | 2009-10-06 | 2011-04-07 | Bela Ban | Dynamic caching of nodes |
| CN102117338A (en) * | 2011-04-02 | 2011-07-06 | 天脉聚源(北京)传媒科技有限公司 | Data base caching method |
| CN102902730A (en) * | 2012-09-10 | 2013-01-30 | 新浪网技术(中国)有限公司 | Method and device for reading data based on data cache |
| CN103870393A (en) * | 2013-07-09 | 2014-06-18 | 携程计算机技术（上海）有限公司 | Cache management method and system |
| CN104572920A (en) * | 2014-12-27 | 2015-04-29 | 北京奇虎科技有限公司 | Data arrangement method and data arrangement device |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106021468A (en) * | 2016-05-17 | 2016-10-12 | 上海携程商务有限公司 | Updating method and system for distributed caches and local caches |
| CN106021468B (en) * | 2016-05-17 | 2019-11-19 | 上海携程商务有限公司 | The update method and system of distributed caching and local cache |
| CN107844386A (en) * | 2016-09-19 | 2018-03-27 | 北京金山云网络技术有限公司 | A kind of data backup, restoration methods and device |
| CN106790562A (en) * | 2016-12-26 | 2017-05-31 | 中国建设银行股份有限公司 | A kind of distributed cache system, method and device |
| CN108628817A (en) * | 2017-03-15 | 2018-10-09 | 腾讯科技（深圳）有限公司 | A kind of data processing method and device |
| CN108628817B (en) * | 2017-03-15 | 2022-07-26 | 腾讯科技（深圳）有限公司 | Data processing method and device |
| CN110334092A (en) * | 2019-06-14 | 2019-10-15 | 福建天泉教育科技有限公司 | Distributed ID generation method, storage medium |
| CN111177196A (en) * | 2019-12-25 | 2020-05-19 | 海尔优家智能科技（北京）有限公司 | Time window aggregation calculation method and device |
| CN111177196B (en) * | 2019-12-25 | 2023-12-19 | 海尔优家智能科技（北京）有限公司 | Time window aggregation calculation method and device |
| CN113626377A (en) * | 2020-05-06 | 2021-11-09 | 杭州海康微影传感科技有限公司 | Bare data storage control method, device and equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105447167B (en) | 2019-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105447167A (en) | Processing method and apparatus for node cache data in distributed system | |
| US9507821B2 (en) | Mail indexing and searching using hierarchical caches | |
| CN101826107B (en) | Hash data processing method and device | |
| US10248680B2 (en) | Index management | |
| CN103019879A (en) | Method and system for processing crash information of browser | |
| CN104572920A (en) | Data arrangement method and data arrangement device | |
| CN103699624A (en) | Browser data processing method and device | |
| CN105095423A (en) | Write-in method and device for count data | |
| CN105095391A (en) | Device and method for identifying organization name by word segmentation program | |
| CN103530420A (en) | Data file dynamic update method and device | |
| CN104216822A (en) | Abnormal information processing method and device | |
| CN103631603A (en) | Software cleaning method and system | |
| CN108427736B (en) | Method for querying data | |
| CN105447021A (en) | Method and system for database query | |
| CN108319634B (en) | Directory access method and device for distributed file system | |
| CN106599247B (en) | Method and device for merging data files in LSM-tree structure | |
| CN111190899A (en) | Buried point data processing method and device, server and storage medium | |
| CN105389394A (en) | Data request processing method and device based on a plurality of database clusters | |
| US8712992B2 (en) | Method and apparatus for web crawling | |
| CN104239538A (en) | Method, system and device for compressing snapshot log | |
| US11520818B2 (en) | Method, apparatus and computer program product for managing metadata of storage object | |
| CN111694992B (en) | Data processing method and device | |
| CN104504330A (en) | Virtual safety detecting method and system | |
| CN105224583A (en) | The method for cleaning of journal file and device | |
| CN103778181A (en) | Method and device for importing icons into favorites of browsers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220722 Address after: Room 801, 8th floor, No. 104, floors 1-19, building 2, yard 6, Jiuxianqiao Road, Chaoyang District, Beijing 100015 Patentee after: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Address before: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park) Patentee before: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Patentee before: Qizhi software (Beijing) Co.,Ltd. |
|
| TR01 | Transfer of patent right |