CN105447167B - The data cached processing method and processing device of distributed system interior joint - Google Patents
The data cached processing method and processing device of distributed system interior joint Download PDFInfo
- Publication number
- CN105447167B CN105447167B CN201510883615.8A CN201510883615A CN105447167B CN 105447167 B CN105447167 B CN 105447167B CN 201510883615 A CN201510883615 A CN 201510883615A CN 105447167 B CN105447167 B CN 105447167B
- Authority
- CN
- China
- Prior art keywords
- data
- data key
- node
- key
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Abstract
The invention discloses a kind of processing method and processing devices that distributed system interior joint is data cached, and wherein method includes: to read at least one data cached, the data cached form with data key values pair of node；It is data cached for each, corresponding data key is extracted, and data key is subjected to persistent storage；When node is restarted, data key is loaded into the caching of node.This programme was greatly reduced to the time needed for data cached progress persistent storage, is reduced the occupancy to system disk resource, is improved loading velocity.
Description
Technical field
The present invention relates to Internet technical fields, and in particular to a kind of processing side that distributed system interior joint is data cached
Method and device.
Background technique
Caching is the buffer area of data exchange, can be first from caching when system needs certain data of reading database
Middle lookup data directly executes, if having found this data in the buffer if do not found in the buffer
This data, then search from database again.Due to the service speed ratio by caching searched from database much faster, because
This caching can rapidly be run with help system, reduce the delay of system.
However when node is restarted, if caching will be lost without passing through persistent storage, caching, and the loss meeting cached
Leading to system, the speed of service is slower whithin a period of time, brings undesirable experience sense to user.Therefore, in the prior art in order to
Cache miss is avoided, usually by data cached carry out persistent storage, however when data cached data volume is larger, caching
The persistent storage of data will not only devote a tremendous amount of time, and can also occupy a large amount of system disk resource.
Summary of the invention
In view of the above problems, it proposes on the present invention overcomes the above problem or at least be partially solved in order to provide one kind
The data cached processing method and processing device of the distributed system interior joint of problem is stated, greatly reduces and is held to data cached
Time needed for longization storage, reduces the occupancy to system disk resource, improve loading velocity.
According to an aspect of the invention, there is provided a kind of processing method that distributed system interior joint is data cached, it should
Method includes:
Read at least one data cached, the data cached form with data key values pair of node；
It is data cached for each, corresponding data key is extracted, and data key is subjected to persistent storage；
When node is restarted, data key is loaded into the caching of node.
According to another aspect of the present invention, a kind of processing unit that distributed system interior joint is data cached is provided, it should
Device includes:
Read module, suitable for reading at least one data cached, the data cached form with data key values pair of node；
Extraction module, it is data cached suitable for being directed to each, extract corresponding data key；
Memory module, the data key suitable for extracting extraction module carry out persistent storage；
First loading module, suitable for when node is restarted, data key is loaded into the caching of node.
The technical solution provided according to the present invention, read node at least one are data cached, wherein data cached to have
The form of data key values pair, it is then data cached for each, corresponding data key is extracted, and data key is carried out persistently
Change storage, finally when node is restarted, data key is loaded into the caching of node.Technical solution provided by the invention is only to slow
Data key in deposit data has carried out persistent storage, to greatly reduce to needed for data cached progress persistent storage
Time, reduce the occupancy to system disk resource, in addition, the technical solution provided according to the present invention, when node is restarted,
Only data key is loaded into the caching of node, improves loading velocity, it is data cached to optimize distributed system interior joint
Processing mode.
The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention,
And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can
It is clearer and more comprehensible, the followings are specific embodiments of the present invention.
Detailed description of the invention
By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field
Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention
Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:
Fig. 1 shows the stream of the data cached processing method of distributed system interior joint according to an embodiment of the invention
Journey schematic diagram；
Fig. 2 shows the data cached processing methods of distributed system interior joint in accordance with another embodiment of the present invention
Flow diagram；
Fig. 3 shows the function of the data cached processing unit of distributed system interior joint according to an embodiment of the invention
It can structural schematic diagram；
Fig. 4 shows the data cached processing unit of distributed system interior joint in accordance with another embodiment of the present invention
Illustrative view of functional configuration.
Specific embodiment
Exemplary embodiments of the present disclosure are described in more detail below with reference to accompanying drawings.Although showing the disclosure in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here
It is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure
It is fully disclosed to those skilled in the art.
Fig. 1 shows the stream of the data cached processing method of distributed system interior joint according to an embodiment of the invention
Journey schematic diagram, as shown in Figure 1, this method comprises the following steps:
Step S100, read node at least one is data cached, the data cached form with data key values pair.
In the step s 100, read node at least one is data cached, wherein data cached with data key values pair
Form, i.e., the data cached form with key-value, wherein key is data key, and value is data value, and data value
It can be the data of diversified forms, be convenient to be managed to data cached by way of data key values pair.
Step S101, it is data cached for each, corresponding data key is extracted, and data key progress persistence is deposited
Storage.
Step S100 have read node at least one it is data cached after, execute step S101, it is slow for each
Deposit data extracts corresponding data key, and data key is carried out persistent storage, that is to say, that in step s101, only mentions
Corresponding data key in data cached is taken out, does not extract corresponding data value, and data key is only subjected to persistence
Storage.Since in most cases, data key length is generally shorter, and the length of data value is long, if will
Data key and data value all carry out persistent storage, will not only devote a tremendous amount of time, but also can occupy a large amount of system magnetic
Disk resource, therefore with data key compared with data value all carries out persistent storage, it is very big that persistent storage only is carried out to data key
Time and the occupancy to system disk resource needed for ground reduces progress persistent storage.
Data key is loaded into the caching of node by step S102 when node is restarted.
When node is restarted, step S102 is executed, the data key through persistent storage is loaded into the caching of node, by
In the caching that only data key need to be loaded into node, it is not necessary that data value is loaded into the caching of node simultaneously, to improve
Loading velocity.
According to the data cached processing method of distributed system interior joint provided in this embodiment, at least the one of node is read
Item is data cached, wherein the data cached form with data key values pair, it is then data cached for each, and it extracts pair
The data key answered, and data key is subjected to persistent storage, finally when node is restarted, data key is loaded into the caching of node
In.Technical solution provided by the invention has only carried out persistent storage to the data key in data cached, to greatly reduce
To the time needed for data cached progress persistent storage, reduce the occupancy to system disk resource, in addition, according to this hair
Data key is only loaded into the caching of node by the technical solution of bright offer when node is restarted, and improves loading velocity, excellent
The data cached processing mode of distributed system interior joint is changed.
Fig. 2 shows the data cached processing methods of distributed system interior joint in accordance with another embodiment of the present invention
Flow diagram, as shown in Fig. 2, this method comprises the following steps:
Step S200, read node at least one is data cached, the data cached form with data key values pair.
Wherein, the data cached form with data key values pair, is convenient to by way of data key values pair to caching
Data are managed.
Step S201, data cached for each, the expired time for extracting corresponding data key and data key is stabbed,
And the flag bit for identifying no data value is set for data key, it is data key values pair by data key and mark bit combination.
Due to only extracting corresponding data key in data cached in step s 201, corresponding data are not extracted
Value, therefore, the corresponding data value of data key are sky, are in step s 201 also data key for the ease of distinguishing these data keies
Flag bit for identifying no data value is set, is then data key values pair by data key and mark bit combination.
In addition, due to can to data cached setting expired time, be also extracted the mistake of data key in step s 201
Phase timestamp.
The expired time of data key values pair and data key is stabbed and carries out persistent storage by step S202.
After step S201 executes completion, the expired time of data key values pair and data key is stabbed and is carried out by step S202
Persistent storage.It, will be by data key and mark hyte since data key values are to being combined by data key and flag bit
The data key values of synthesis to carry out persistent storage, with data key compared with data value all carries out persistent storage, reduce into
Time needed for row persistent storage.In addition, being stabbed according to the expired time of data key, logarithm for the ease of in the next steps
Judged according to whether key is expired, also needs to carry out persistent storage to the expired time stamp of data key in step S202.
Step S203, when node is restarted, by data key values in the caching for being loaded into node.
When node is restarted, step S203 is by the data key values being combined by data key and flag bit to being loaded into node
In caching, due to being not necessarily to that data value is loaded into the caching of node, to improve loading velocity.
Step S204 stabs according to the expired time of data key when node is restarted, judges whether data key is out of date；If
It is to then follow the steps S205；If it is not, thening follow the steps S206.
When node is restarted, step S204 is stabbed according to the expired time of data key, judges whether data key is out of date；If sentencing
Disconnected data key out is out of date, thens follow the steps S205, deletes the data key；If judging, data key is not out of date, thens follow the steps
S206。
Step S205 deletes data key.
If judging that data key is out of date through step S204, deletion has expired data key.
Step S206 receives the update message for the data key values pair that other nodes are sent.
After node is restarted, if there is other nodes send the update message of data key values pair, then executing step
S206 receives the update message for the data key values pair that other nodes are sent.
Step S207 sends out other nodes are updated to data key values corresponding in the caching of node according to update message
The data key values pair sent.
In step S207, according to the received update message of step S206 institute, by data key corresponding in the caching of node
Value is to the data key values pair for being updated to other nodes transmissions.
For example, in step S206, have received that other nodes send about data key values to 1 update message, and
According to update message it is found that data key values are updated to data value 1 to the data value in 1, then in step S207, by node
Corresponding data key values are updated to the data key values of other nodes transmissions to 1 to 1 in caching.It is assumed that corresponding in the caching of node
Data key values be not updated to 1,, will then in step S207 then the data key values are to having flag bit in 1
Corresponding data key values are updated to data key values that other nodes are sent to the number in 1 to the flag bit in 1 in the caching of node
According to value 1.
In addition, if this method may also include that the data cached access request received for node, according to being accessed
Data key search the corresponding data value of data key stored in the cachings of other nodes, data value is loaded into the caching of node
In, and the step of responding access request returned data value.Specifically, it can be realized by step S208 to step S210.
Step S208 judges to load in the caching of node if receiving the data cached access request for node
The data key values accessed to whether include flag bit；If so, thening follow the steps S209；If it is not, thening follow the steps S210.
Due to being accessed of when receiving the data cached access request for node, being loaded in the caching of node
Data key values are not to that may be updated, therefore, in step S208, need what is loaded in the caching to node to be accessed
Data key values to whether judging comprising flag bit.If it is judged that the data key values accessed are to comprising flag bit, then
Execute step S209；If it is judged that the data key values accessed then follow the steps S210 to flag bit is not included.
Step S209 searches the corresponding data of data key stored in the caching of other nodes according to the data key accessed
Value, and the flag bit being arranged for data key is replaced with into data value.
If judging that accessed data key values to comprising flag bit, represent accessed data key values through step S208
Centering no data value, then need to be implemented step S209, is searched according to the data key accessed and is stored in the caching of other nodes
The flag bit being arranged for data key is replaced with data value by the corresponding data value of data key, so that the data key values centering be made to have
There is corresponding data value.After the completion of step S209 is executed, step S210 is then executed.
Step S210, the data value that response access request returned data key-value pair includes.
If judging that accessed data key values to flag bit is not included, represent accessed data key through step S208
Being worth centering has corresponding data value, then can directly execute step S210, and response access request returned data key-value pair includes
Data value.Alternatively, having data key values centering after the flag bit being arranged for data key is replaced with data value by step S209
There is corresponding data value, then then step S210, response access request return to the data value.
According to the data cached processing method of distributed system interior joint provided in this embodiment, number is cached for each
According to not only extracting corresponding data key, the expired time stamp of data key also extracted, so as to when node is restarted, according to number
It is stabbed according to the expired time of key, deletion has expired data key；For the ease of distinguishing, it is arranged for data key for identifying no data value
Flag bit, be data key values pair by data key and mark bit combination, then when node is restarted, by data key values to being loaded into
In the caching of node, and the update message of the data key values pair sent according to other nodes, it will be corresponding in the caching of node
Data key values are to the data key values pair for being updated to other nodes transmissions.
In addition, the technical solution provided according to the present invention, after receiving the data cached access request for node,
Only when judging accessed data key values to comprising flag bit, other nodes are just searched according to the data key accessed
The flag bit being arranged for data key is replaced with data value, and responds access by the corresponding data value of the data key stored in caching
Request returned data value；When judging accessed data key values to flag bit is not included, then returned directly in response to access request
Return data key values to comprising data value.The technical solution provided according to the present invention not only ensure that when node is restarted, and cache
Data are not lost, and are also greatly reduced to the time needed for data cached progress persistent storage, are reduced to system disk
The occupancy of resource improves the loading velocity data cached when node is restarted, and is only judging accessed data
When key-value pair includes flag bit, just needs search the data key pair stored in the cachings of other nodes according to the data key accessed
The data value answered has advanced optimized the data cached processing mode of distributed system interior joint to improve response speed.
Fig. 3 shows the function of the data cached processing unit of distributed system interior joint according to an embodiment of the invention
Can structural schematic diagram, as shown in figure 3, the device include: read module 301, extraction module 302, memory module 303 and first plus
Carry module 304.
Read module 301 is suitable for: read node at least one is data cached, the data cached shape with data key values pair
Formula.
At least one of the reading node of read module 301 is data cached, wherein the data cached shape with data key values pair
Formula, i.e., the data cached form with key-value, wherein key is data key, and value is data value, and data value can
For the data of diversified forms, it is convenient to be managed to data cached by way of data key values pair.
Extraction module 302 is suitable for: it is data cached for each, extract corresponding data key.
Read module 301 have read node at least one it is data cached after, trigger extraction module 302, for every
One data cached, extracts corresponding data key, that is to say, that only extracts data cached middle correspondence in extraction module 302
Data key, do not extract corresponding data value.
Memory module 303 is suitable for: the data key that extraction module 301 is extracted carries out persistent storage.
Since extraction module 302 only extracts corresponding data key in data cached, corresponding data are not extracted
Value, and in most cases, data key length is generally shorter, and the length of data value is long, if will count
Persistent storage is all carried out according to key and data value, will not only be devoted a tremendous amount of time, but also a large amount of system disk can be occupied
Resource, therefore with data key compared with data value all carries out persistent storage, memory module 303 only carries out persistence to data key
Storage greatly reduces the time needed for carrying out persistent storage, and reduces the occupancy of system disk resource.
First loading module 304 is suitable for: when node is restarted, data key being loaded into the caching of node.
When node is restarted, the first loading module 304 is triggered, the data key of stored 303 persistent storage of module is added
It is downloaded in the caching of node, since data key need to be only loaded into the caching of node by the first loading module 304, without while by
Data value is loaded into the caching of node, to improve loading velocity.
According to the data cached processing unit of distributed system interior joint provided in this embodiment, read module reads node
It is at least one data cached, wherein the data cached form with data key values pair, then extraction module is slow for each
Deposit data extracts corresponding data key, and data key is carried out persistent storage by memory module, finally when node is restarted, the
Data key is loaded into the caching of node by one loading module.Technical solution provided by the invention is only to the data in data cached
Key has carried out persistent storage, to greatly reduce to the time needed for data cached progress persistent storage, reduces
To the occupancy of system disk resource, in addition, the technical solution provided according to the present invention only adds data key when node is restarted
It is downloaded in the caching of node, improves loading velocity, optimize the data cached processing mode of distributed system interior joint.
Fig. 4 shows the data cached processing unit of distributed system interior joint in accordance with another embodiment of the present invention
Illustrative view of functional configuration, as shown in figure 4, the device includes: read module 401, extraction module 402, memory module 403, first
Loading module 404, access modules 405, the second loading module 406 and respond module 407.
Read module 401 is suitable for: read node at least one is data cached, the data cached shape with data key values pair
Formula.
Extraction module 402 is suitable for: it is data cached for each, extract corresponding data key.
Read module 401 have read node at least one it is data cached after, trigger extraction module 402, for every
One data cached, extracts corresponding data key, does not extract corresponding data value.
Memory module 403 is suitable for: data key is carried out persistent storage.
It is data cached for each in extraction module 402, after extracting corresponding data key, trigger memory module
403, the data key extracted is subjected to persistent storage.
First loading module 404 is suitable for: when node is restarted, data key being loaded into the caching of node.
When node is restarted, the first loading module 404 is triggered, the data key of 403 persistent storages of memory module is added
It is downloaded in the caching of node.
Access modules 405 are suitable for: receiving the data cached access request for being directed to node.
Second loading module 406 is suitable for: if access modules 405 receive the data cached access request for node,
The corresponding data value of data key stored in the caching of other nodes is then searched according to the data key accessed, and data value is loaded
Into the caching of node.
Respond module 407 is suitable for: response access request returned data value.
After in the caching that data value is loaded into node by the second loading module 406, respond module 407, response are triggered
Access request returned data value.
Since extraction module 402 only extracts corresponding data key in data cached, corresponding data are not extracted
Value, therefore, the corresponding data value of data key are sky, for the ease of distinguishing these data keies, the device further include: setup module
408, it is suitable for data key and flag bit for identifying no data value is set, be data key values by data key and mark bit combination
It is right.
In the case where the flag bit for identifying no data value is arranged for data key, memory module 403 is further adapted for:
By data key values to progress persistent storage；First loading module 404 is further adapted for: when node is restarted, by data key values
To in the caching for being loaded into node.
After node is restarted, it is possible to which other nodes can send the update message of data key values pair, so the device is also
Can further comprise: update module 409, suitable for receiving the update message for the data key values pair that other nodes are sent, according to update
Message, by data key values corresponding in the caching of node to the data key values pair for being updated to other nodes transmissions.
Due to being accessed of when receiving the data cached access request for node, being loaded in the caching of node
Data key values are not to may be updated, therefore the device further include: first judgment module 410, if being suitable for access modules
405 receive the data cached access request for node, the data key values accessed for judging to load in the caching of node
To whether include flag bit.
At this point, the second loading module 406 is further adapted for: if first judgment module 410 judges accessed data key
Value searches the corresponding data of data key stored in the caching of other nodes according to the data key accessed to comprising flag bit
Value, and the flag bit being arranged for data key is replaced with into data value.In the mark that the second loading module 406 will be arranged for data key
After position replaces with data value, respond module 407 is triggered, responds access request returned data value.
Respond module 407 is further adapted for: if first judgment module 410 judges accessed data key values to not including
Flag bit, the data value that response access request returned data key-value pair includes.
In addition, due to can to data cached setting expired time, the extraction module 402 in the device is further fitted
In: the expired time stamp of data key is extracted from data cached；Memory module 403 is further adapted for: by the expired of data key
Timestamp carries out persistent storage.
In order to judge according to whether the expired time of data key stamp is expired to data key, and to having expired data key
It is handled, the device further include: the second judgment module 411 and removing module 412.
Wherein, the second judgment module 411 is suitable for: when node is restarted, being stabbed according to the expired time of data key, judges data
Whether key is out of date.If the second judgment module 411 judges that data key is out of date, removing module 412 is triggered, deletes data
Key.
According to the data cached processing unit of distributed system interior joint provided in this embodiment, number is cached for each
According to extraction module not only extracts corresponding data key, extracts the expired time stamp of data key, also to restart when node
When, the second judgment module is stabbed according to the expired time of data key, judges whether data key is out of date；For the ease of distinguishing, setting
Module is that the flag bit for identifying no data value is arranged in data key, is data key values pair by data key and mark bit combination, so
Afterwards when node is restarted, the first loading module is by data key values in the caching for being loaded into node, and update module is according to it
The update message for the data key values pair that its node is sent, by data key values corresponding in the caching of node to being updated to other nodes
The data key values pair of transmission.
In addition, the technical solution provided according to the present invention, receives the data cached visit for node in access modules
After asking request, only when first judgment module judges accessed data key values to comprising flag bit, just according to being accessed
Data key search the corresponding data value of data key stored in the cachings of other nodes, the flag bit being arranged for data key is replaced
It is changed to data value, and responds access request returned data value；Judge accessed data key values to not in first judgment module
When comprising flag bit, then respond module is directly triggered.The technical solution provided according to the present invention not only ensure that restarts in node
When, it is data cached not lose, it also greatly reduces to the time needed for data cached progress persistent storage, reduces to being
The occupancy of system disk resource, improves the loading velocity data cached when node is restarted, and only judging to be accessed
Data key values to comprising flag bit when, just need to search the number stored in the cachings of other nodes according to the data key accessed
According to the corresponding data value of key, to improve response speed, the data cached place of distributed system interior joint has been advanced optimized
Reason mode.
Algorithm and display are not inherently related to any particular computer, virtual system, or other device provided herein.
Various general-purpose systems can also be used together with teachings based herein.As described above, it constructs required by this kind of system
Structure be obvious.In addition, the present invention is also not directed to any particular programming language.It should be understood that can use various
Programming language realizes summary of the invention described herein, and the description done above to language-specific is to disclose this hair
Bright preferred forms.
In the instructions provided here, numerous specific details are set forth.It is to be appreciated, however, that implementation of the invention
Example can be practiced without these specific details.In some instances, well known method, structure is not been shown in detail
And technology, so as not to obscure the understanding of this specification.
Similarly, it should be understood that in order to simplify the disclosure and help to understand one or more of the various inventive aspects,
Above in the description of exemplary embodiment of the present invention, each feature of the invention is grouped together into single implementation sometimes
In example, figure or descriptions thereof.However, the disclosed method should not be interpreted as reflecting the following intention: i.e. required to protect
Shield the present invention claims features more more than feature expressly recited in each claim.More precisely, as following
Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore,
Thus the claims for following specific embodiment are expressly incorporated in the specific embodiment, wherein each claim itself
All as a separate embodiment of the present invention.
Those skilled in the art will understand that can be carried out adaptively to the module in the equipment in embodiment
Change and they are arranged in one or more devices different from this embodiment.It can be the module or list in embodiment
Member or component are combined into a module or unit or component, and furthermore they can be divided into multiple submodule or subelement or
Sub-component.Other than such feature and/or at least some of process or unit exclude each other, it can use any
Combination is to all features disclosed in this specification (including adjoint claim, abstract and attached drawing) and so disclosed
All process or units of what method or apparatus are combined.Unless expressly stated otherwise, this specification is (including adjoint power
Benefit require, abstract and attached drawing) disclosed in each feature can carry out generation with an alternative feature that provides the same, equivalent, or similar purpose
It replaces.
In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments
In included certain features rather than other feature, but the combination of the feature of different embodiments mean it is of the invention
Within the scope of and form different embodiments.For example, in the following claims, embodiment claimed is appointed
Meaning one of can in any combination mode come using.
Various component embodiments of the invention can be implemented in hardware, or to run on one or more processors
Software module realize, or be implemented in a combination thereof.It will be understood by those of skill in the art that can be used in practice
Microprocessor or digital signal processor (DSP) realize one of some or all components according to embodiments of the present invention
A little or repertoire.The present invention is also implemented as setting for executing some or all of method as described herein
Standby or program of device (for example, computer program and computer program product).It is such to realize that program of the invention deposit
Storage on a computer-readable medium, or may be in the form of one or more signals.Such signal can be from because of spy
It downloads and obtains on net website, be perhaps provided on the carrier signal or be provided in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and ability
Field technique personnel can be designed alternative embodiment without departing from the scope of the appended claims.In the claims,
Any reference symbol between parentheses should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not
Element or step listed in the claims.Word "a" or "an" located in front of the element does not exclude the presence of multiple such
Element.The present invention can be by means of including the hardware of several different elements and being come by means of properly programmed computer real
It is existing.In the unit claims listing several devices, several in these devices can be through the same hardware branch
To embody.The use of word first, second, and third does not indicate any sequence.These words can be explained and be run after fame
Claim.
The invention discloses:
A1, a kind of processing method that distributed system interior joint is data cached, which comprises
Read at least one data cached, the data cached form with data key values pair of node；
It is data cached for each, corresponding data key is extracted, and the data key is subjected to persistent storage；
When the node is restarted, the data key is loaded into the caching of node.
A2, method according to a1, wherein when the node is restarted and the data key is loaded into the caching of node
Later, the method also includes:
If receiving the data cached access request for node, other nodes are searched according to the data key accessed
Caching in the corresponding data value of the data key that stores, the data value is loaded into the caching of node, and respond institute
It states access request and returns to the data value.
A3, method according to a1 or a2, wherein it is data cached for each, corresponding data key is extracted, and
The step of data key is carried out persistent storage further comprises:
The flag bit for identifying no data value is set for the data key, is number by the data key and mark bit combination
According to key-value pair；
By data key values to progress persistent storage.
A4, method according to a3, wherein when the node is restarted, the data key is loaded into the slow of node
Step in depositing further comprises:
When the node is restarted, by the data key values in the caching for being loaded into node.
A5, the method according to A3 or A4, wherein when the node is restarted and the data key is loaded into node
After caching, the method also includes:
If receiving the data cached access request for node, judge that is loaded in the caching of the node is accessed
Data key values to whether include the flag bit；
If judging, accessed data key values to comprising the flag bit, are searched other according to the data key accessed
The corresponding data value of the data key stored in the caching of node, and the flag bit being arranged for the data key is replaced with into institute
State data value；
If judging accessed data key values to the flag bit is not included, respond described in the access request return
Data key values to comprising data value.
A6, according to the described in any item methods of A1-A5, wherein for each it is data cached, extract corresponding data
Key, and the step of data key is carried out persistent storage further comprises:
From it is described it is data cached in extract the expired time stamp of data key and the data key, and by the data key
And the expired time stamp of the data key carries out persistent storage.
A7, the method according to A6, wherein when the node is restarted, the method also includes: according to the data
The expired time of key is stabbed, and judges whether the data key is out of date；If so, deleting the data key；
If judging that the data key is not out of date and receiving the data cached access request for node, according to institute
The data key of access searches the corresponding data value of the data key stored in the cachings of other nodes.
A8, according to the described in any item methods of A1-A7, wherein the method also includes:
Receive the update message for the data key values pair that other nodes are sent；
According to the update message, by data key values corresponding in the caching of the node to being updated to other nodes
The data key values pair of transmission.
B9, a kind of processing unit that distributed system interior joint is data cached, described device include:
Read module, it is described data cached with data key values pair suitable for reading at least one data cached of node
Form；
Extraction module, it is data cached suitable for being directed to each, extract corresponding data key；
Memory module, the data key suitable for extracting the extraction module carry out persistent storage；
First loading module, suitable for when the node is restarted, the data key is loaded into the caching of node.
B10, the device according to B9, wherein described device further include:
Access modules, suitable for receiving the data cached access request for being directed to node；
Second loading module, if receiving the data cached access request for node suitable for the access modules,
The corresponding data value of the data key stored in the caching of other nodes is searched according to the data key accessed, by the data
Value is loaded into the caching of node；
Respond module is adapted to respond to the access request and returns to the data value.
B11, device according to b10, wherein described device further include: setup module is suitable for the data key and sets
The flag bit for identifying no data value is set, is data key values pair by the data key and mark bit combination；
The memory module is further adapted for: by data key values to progress persistent storage.
B12, the device according to B11, wherein first loading module is further adapted for: when the node is restarted
When, by the data key values in the caching for being loaded into node.
B13, the device according to B11 or B12, wherein described device further include: first judgment module, if being suitable for institute
It states access modules and receives data cached access request for node, judge that is loaded in the caching of the node is accessed
Data key values to whether include the flag bit；
Second loading module is further adapted for: if the first judgment module judges accessed data key values pair
Comprising the flag bit, the corresponding number of the data key stored in the caching of other nodes is searched according to the data key accessed
The data value is replaced with according to value, and by the flag bit being arranged for the data key；
The respond module is further adapted for: if the first judgment module judges accessed data key values to not wrapping
Containing the flag bit, respond the access request return the data key values to comprising data value.
B14, according to the described in any item devices of B10-B13, wherein the extraction module is further adapted for: from described slow
The expired time stamp of data key and the data key is extracted in deposit data；
The memory module is further adapted for: it is lasting that the expired time of the data key and data key stamp is carried out
Change storage.
B15, device according to b14, wherein described device includes: the second judgment module and removing module；
Second judgment module, suitable for being stabbed according to the expired time of the data key when the node is restarted, judgement
Whether the data key is out of date；If second judgment module judges that the data key is out of date, the deletion is triggered
Module；
The removing module is suitable for deleting the data key.
B16, according to the described in any item devices of B9-B15, wherein described device further include: update module, be suitable for receive
The update message for the data key values pair that other nodes are sent will be corresponding in the caching of the node according to the update message
Data key values are to the data key values pair for being updated to other node transmissions.
Claims (14)
1. a kind of processing method that distributed system interior joint is data cached, which comprises
Read at least one data cached, the data cached form with data key values pair of node；
It is data cached for each, corresponding data key is extracted, and the data key is subjected to persistent storage；
When the node is restarted, the data key is loaded into the caching of node；
Wherein, described data cached for each, corresponding data key is extracted, and data key progress persistence is deposited
The step of storage, further comprises: the flag bit for identifying no data value is arranged for the data key, by the data key and mark
Will hyte is combined into data key values pair；By data key values to progress persistent storage.
2. according to the method described in claim 1, wherein, when the node is restarted and the data key is loaded into the slow of node
After depositing, the method also includes:
If receiving the data cached access request for node, the slow of other nodes is searched according to the data key accessed
The corresponding data value of the data key for depositing middle storage, the data value is loaded into the caching of node, and respond the visit
Ask that request returns to the data value.
3. according to the method described in claim 1, wherein, when the node is restarted, the data key is loaded into node
Step in caching further comprises:
When the node is restarted, by the data key values in the caching for being loaded into node.
4. according to the method described in claim 1, wherein, when the node is restarted and the data key is loaded into the slow of node
After depositing, the method also includes:
If receiving the data cached access request for node, the number accessed loaded in the caching of the node is judged
It whether include the flag bit according to key-value pair；
If judging, accessed data key values to comprising the flag bit, search other nodes according to the data key accessed
Caching in the corresponding data value of the data key that stores, and the flag bit being arranged for the data key is replaced with into the number
According to value；
If judging, accessed data key values to the flag bit is not included, respond the access request and return to the data
The data value that key-value pair includes.
5. method according to claim 1-4, wherein it is data cached for each, extract corresponding number
According to key, and the step of data key is carried out persistent storage further comprises:
From it is described it is data cached in extract the expired time stamp of data key and the data key, and by the data key and
The expired time stamp of the data key carries out persistent storage.
6. according to the method described in claim 5, wherein, when the node is restarted, the method also includes: according to the number
It is stabbed according to the expired time of key, judges whether the data key is out of date；If so, deleting the data key；
If judging that the data key is not out of date and receiving the data cached access request for node, according to being accessed
Data key search the corresponding data value of the data key stored in the cachings of other nodes.
7. method according to claim 1-4, wherein the method also includes:
Receive the update message for the data key values pair that other nodes are sent；
According to the update message, data key values corresponding in the caching of the node are sent to other nodes are updated to
Data key values pair.
8. a kind of processing unit that distributed system interior joint is data cached, described device include:
Read module, suitable for reading at least one data cached, the data cached form with data key values pair of node；
Extraction module, it is data cached suitable for being directed to each, extract corresponding data key；
Memory module, the data key suitable for extracting the extraction module carry out persistent storage；
First loading module, suitable for when the node is restarted, the data key is loaded into the caching of node；
Described device further include: setup module is suitable for the data key and flag bit for identifying no data value is arranged, by institute
Stating data key and mark bit combination is data key values pair；
The memory module is further adapted for: by data key values to progress persistent storage.
9. device according to claim 8, wherein described device further include:
Access modules, suitable for receiving the data cached access request for being directed to node；
Second loading module, if receiving the data cached access request for node, basis suitable for the access modules
The data key accessed searches the corresponding data value of the data key stored in the cachings of other nodes, and the data value is added
It is downloaded in the caching of node；
Respond module is adapted to respond to the access request and returns to the data value.
10. device according to claim 8, wherein first loading module is further adapted for: when the node is restarted
When, by the data key values in the caching for being loaded into node.
11. device according to claim 9, wherein described device further include: first judgment module, if being suitable for the visit
It asks that module receives the data cached access request for node, judges the number accessed loaded in the caching of the node
It whether include the flag bit according to key-value pair；
Second loading module is further adapted for: if the first judgment module judge accessed data key values to comprising
The flag bit searches the corresponding data of the data key stored in the caching of other nodes according to the data key accessed
Value, and the flag bit being arranged for the data key is replaced with into the data value；
The respond module is further adapted for: if the first judgment module judges accessed data key values to not comprising institute
State flag bit, respond the access request return the data key values to comprising data value.
12. according to the described in any item devices of claim 8-11, wherein the extraction module is further adapted for: delaying from described
The expired time stamp of data key and the data key is extracted in deposit data；
The memory module is further adapted for: the expired time of the data key and the data key being stabbed progress persistence and is deposited
Storage.
13. device according to claim 12, wherein described device includes: the second judgment module and removing module；
Second judgment module, suitable for being stabbed according to the expired time of the data key when the node is restarted, described in judgement
Whether data key is out of date；If second judgment module judges that the data key is out of date, the removing module is triggered；
The removing module is suitable for deleting the data key.
14. according to the described in any item devices of claim 8-11, wherein described device further include: update module is suitable for receiving
The update message for the data key values pair that other nodes are sent will be corresponding in the caching of the node according to the update message
Data key values are to the data key values pair for being updated to other node transmissions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510883615.8A CN105447167B (en) | 2015-12-04 | 2015-12-04 | The data cached processing method and processing device of distributed system interior joint |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510883615.8A CN105447167B (en) | 2015-12-04 | 2015-12-04 | The data cached processing method and processing device of distributed system interior joint |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105447167A CN105447167A (en) | 2016-03-30 |
| CN105447167B true CN105447167B (en) | 2019-03-01 |
Family
ID=55557343
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510883615.8A Active CN105447167B (en) | 2015-12-04 | 2015-12-04 | The data cached processing method and processing device of distributed system interior joint |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105447167B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106021468B (en) * | 2016-05-17 | 2019-11-19 | 上海携程商务有限公司 | The update method and system of distributed caching and local cache |
| CN107844386B (en) * | 2016-09-19 | 2020-11-27 | 北京金山云网络技术有限公司 | Data backup and recovery method and device |
| CN106790562A (en) * | 2016-12-26 | 2017-05-31 | 中国建设银行股份有限公司 | A kind of distributed cache system, method and device |
| CN108628817B (en) * | 2017-03-15 | 2022-07-26 | 腾讯科技（深圳）有限公司 | Data processing method and device |
| CN110334092A (en) * | 2019-06-14 | 2019-10-15 | 福建天泉教育科技有限公司 | Distributed ID generation method, storage medium |
| CN111177196B (en) * | 2019-12-25 | 2023-12-19 | 海尔优家智能科技（北京）有限公司 | Time window aggregation calculation method and device |
| CN113626377A (en) * | 2020-05-06 | 2021-11-09 | 杭州海康微影传感科技有限公司 | Bare data storage control method, device and equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117338A (en) * | 2011-04-02 | 2011-07-06 | 天脉聚源(北京)传媒科技有限公司 | Data base caching method |
| CN102902730A (en) * | 2012-09-10 | 2013-01-30 | 新浪网技术(中国)有限公司 | Method and device for reading data based on data cache |
| CN103870393A (en) * | 2013-07-09 | 2014-06-18 | 携程计算机技术（上海）有限公司 | Cache management method and system |
| CN104572920A (en) * | 2014-12-27 | 2015-04-29 | 北京奇虎科技有限公司 | Data arrangement method and data arrangement device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8914457B2 (en) * | 2009-10-06 | 2014-12-16 | Red Hat, Inc. | Caching of nodes in cache cluster |
-
2015
- 2015-12-04 CN CN201510883615.8A patent/CN105447167B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102117338A (en) * | 2011-04-02 | 2011-07-06 | 天脉聚源(北京)传媒科技有限公司 | Data base caching method |
| CN102902730A (en) * | 2012-09-10 | 2013-01-30 | 新浪网技术(中国)有限公司 | Method and device for reading data based on data cache |
| CN103870393A (en) * | 2013-07-09 | 2014-06-18 | 携程计算机技术（上海）有限公司 | Cache management method and system |
| CN104572920A (en) * | 2014-12-27 | 2015-04-29 | 北京奇虎科技有限公司 | Data arrangement method and data arrangement device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105447167A (en) | 2016-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105447167B (en) | The data cached processing method and processing device of distributed system interior joint | |
| CN109597818A (en) | Data-updating method, device, storage medium and equipment | |
| CN105608159B (en) | The method and apparatus of data buffer storage | |
| CN107943718B (en) | Method and device for cleaning cache file | |
| CN106294352B (en) | A kind of document handling method, device and file system | |
| CN105515872B (en) | The update method of configuration information, apparatus and system | |
| CN108228799B (en) | Object index information storage method and device | |
| CN104133783B (en) | Method and device for processing distributed cache data | |
| CN107807797A (en) | The method, apparatus and server of data write-in | |
| CN102968456B (en) | A kind of raster data reading and processing method and device | |
| CN106919628A (en) | A kind for the treatment of method and apparatus of diagram data | |
| CN104899077A (en) | Process information acquiring method and device based on container technology | |
| CN105574151B (en) | Image storage method and equipment | |
| CN107368563B (en) | Database data deleting method and device, electronic equipment and storage medium | |
| CN106599247B (en) | Method and device for merging data files in LSM-tree structure | |
| CN108154024A (en) | A kind of data retrieval method, device and electronic equipment | |
| CN103761240A (en) | Database access method and database access device | |
| CN105159545A (en) | Method and device for processing popup box on intelligent terminal for test | |
| CN106970998A (en) | The update method and device of news data | |
| CN109190367A (en) | Utilize the method and device of sandbox operation application program installation kit | |
| CN105389394A (en) | Data request processing method and device based on a plurality of database clusters | |
| CN102945343A (en) | Method and device for enumerating system process | |
| CN103336671A (en) | Method and equipment for acquiring data from network | |
| CN104239538A (en) | Method, system and device for compressing snapshot log | |
| CN104504330B (en) | Virtualize safety detection method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220722 Address after: Room 801, 8th floor, No. 104, floors 1-19, building 2, yard 6, Jiuxianqiao Road, Chaoyang District, Beijing 100015 Patentee after: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Address before: 100088 room 112, block D, 28 new street, new street, Xicheng District, Beijing (Desheng Park) Patentee before: BEIJING QIHOO TECHNOLOGY Co.,Ltd. Patentee before: Qizhi software (Beijing) Co.,Ltd. |