CN107231395A - Date storage method, device and system - Google Patents
Date storage method, device and system Download PDFInfo
- Publication number
- CN107231395A CN107231395A CN201610178110.6A CN201610178110A CN107231395A CN 107231395 A CN107231395 A CN 107231395A CN 201610178110 A CN201610178110 A CN 201610178110A CN 107231395 A CN107231395 A CN 107231395A
- Authority
- CN
- China
- Prior art keywords
- data
- local cache
- remote distributed
- distributed caching
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
- H04L67/63—Routing a service request depending on the request content or context
Abstract
The application proposes a kind of date storage method, device and system, and the date storage method includes：Receive the read request for reading data；The corresponding data of the read request are inquired about in local cache and remote distributed caching；If all not storing the data in local cache and remote distributed caching, the data are read from database；After the data are read, the data are write in local cache and remote distributed caching.This method can be stored data in local cache and remote distributed caching, it is to avoid data access request is focused on into request wait or the timeout issue that remote distributed caching is caused, it is ensured that business trouble-free operation.
Description
Technical field
The application is related to Internet technical field, more particularly to a kind of date storage method, device and system.
Background technology
In the case where read access amount is more fewer than larger, write access amount, introducing distributed caching can be reduced
Access to backend application or database, largely reduces rear end cost.But, for some ratios
The data access request relatively concentrated, these requests can concentrate certain back end having access in distributed caching
Server, under limited network interface card data transmission capabilities, it is easy to cause this number of units according to the network interface card of node server
Overload.
In correlation technique, because central access causes after certain number of units overloads according to node server, cache client
The access request to this number of units according to node server can be blocked within ensuing a period of time, can be caused follow-up
The request for reaching the back end server occurs waiting or time-out, and influence is brought on the availability of business.
The content of the invention
The application is intended at least solve one of technical problem in correlation technique to a certain extent.
Therefore, the purpose of the application is to propose a kind of date storage method, this method can be by data
It is stored in local cache and remote distributed caching, it is to avoid data access request is focused on into remote distributed
Cache the request wait caused or timeout issue, it is ensured that business trouble-free operation.
Further object is to propose a kind of data storage device.
Further object is to propose a kind of data-storage system.
To reach above-mentioned purpose, the date storage method that the application first aspect embodiment is proposed, including：Connect
Receive the read request for reading data；The read request pair is inquired about in local cache and remote distributed caching
The data answered；If the data are not all stored in local cache and remote distributed caching, from data
The data are read in storehouse；After the data are read, the data are write into local cache and long-range point
In cloth caching.
The application first aspect embodiment propose date storage method, by write data into local cache and
Remote distributed is cached, it is possible to achieve L2 cache, so as to first can locally delay when needing and reading data
Deposit middle reading, it is to avoid focus on and read in remote distributed caching, it is to avoid the back end service of some focus
The request of device occurs waiting or time-out, it is to avoid influence is brought on service availability, it is ensured that business trouble-free operation.
To reach above-mentioned purpose, the data storage device that the application second aspect embodiment is proposed, including：Connect
Module is received, for receiving the read request for being used for reading data；Enquiry module, in local cache and remotely
The corresponding data of the read request are inquired about in distributed caching；Read module, if for local cache and far
The data are not all stored in journey distributed caching, then the data are read from database；Writing module,
For after the data are read, the data to be write in local cache and remote distributed caching.
The application second aspect embodiment propose data storage device, by write data into local cache and
Remote distributed is cached, it is possible to achieve L2 cache, so as to first can locally delay when needing and reading data
Deposit middle reading, it is to avoid focus on and read in remote distributed caching, it is to avoid the back end service of some focus
The request of device occurs waiting or time-out, it is to avoid influence is brought on service availability, it is ensured that business trouble-free operation.
To reach above-mentioned purpose, the data-storage system that the application third aspect embodiment is proposed, including：This
Apply for the data storage device that second aspect embodiment is proposed, and, local cache and remote distributed caching,
The local cache and remote distributed are cached for data storage.
The application third aspect embodiment propose data-storage system, by write data into local cache and
Remote distributed is cached, it is possible to achieve L2 cache, so as to first can locally delay when needing and reading data
Deposit middle reading, it is to avoid focus on and read in remote distributed caching, it is to avoid the back end service of some focus
The request of device occurs waiting or time-out, it is to avoid influence is brought on service availability, it is ensured that business trouble-free operation.
The aspect and advantage that the application is added will be set forth in part in the description, and will partly be retouched from following
Become obvious in stating, or recognize by the practice of the application.
Brief description of the drawings
The above-mentioned and/or additional aspect of the application and advantage are from the following description of the accompanying drawings of embodiments
It will be apparent and be readily appreciated that, wherein：
Fig. 1 is the schematic flow sheet for the data cache method that the application one embodiment is proposed；
Fig. 2 is the schematic flow sheet of the date storage method of the application another embodiment proposition；
Fig. 3 is the corresponding system interaction schematic diagrames of Fig. 2；
Fig. 4 be the application one embodiment propose Stale Cache in data schematic flow sheet；
Fig. 5 is the corresponding system interaction schematic diagrames of Fig. 4；
Fig. 6 is the structural representation for the data storage device that the application one embodiment is proposed；
Fig. 7 is the structural representation of the data storage device of the application another embodiment proposition；
Fig. 8 is the structural representation for the data-storage system that the application one embodiment is proposed.
Embodiment
Embodiments herein is described below in detail, the example of the embodiment is shown in the drawings, wherein certainly
Beginning to same or similar label eventually represents same or similar module or the mould with same or like function
Block.The embodiments described below with reference to the accompanying drawings are exemplary, is only used for explaining the application, and can not
It is interpreted as the limitation to the application.On the contrary, embodiments herein includes falling into attached claims
All changes, modification and equivalent in the range of spirit and intension.
Fig. 1 is the schematic flow sheet for the data cache method that the application one embodiment is proposed.
The present embodiment is by taking the execution flow of application server as an example.
Referring to Fig. 1, the method for the present embodiment includes：
S11：Receive the read request for reading data.
For example, when called side needs to read data, can be sent out to application server (APP server)
Send read request.
S12：The corresponding data of the read request are inquired about in local cache and remote distributed caching.
Wherein, data can correspond to the mark of data storage and the data of particular content, read request in storage
In can carry the mark (data id) of data, so as in the data of storage inquiry with the presence or absence of read
The mark carried in request, and, when storing data id according to above-mentioned corresponding relation, acquisition pair
The data answered.
Application server can first be inquired about in local cache and cached again in remote distributed when inquiring about data
Middle inquiry.For example, called side can carry data id when needing to read data in read request, should
Received with server after read request, can first be inquired about in local cache and whether store data id, such as
Fruit does not store, then inquiry can whether there is data id in remote distributed caching, if also do not had
There is storage, it may be determined that local cache and remote distributed caching all do not store the number that called side needs to read
According to.
Further, data that can be to frequency of use less than preset value in local cache carry out expired processing,
For example, using (Least Recently Used, LRU) algorithm to carry out expired processing using minimum in the recent period.
Wherein, during LRU memory management a kind of page replacement algorithm, in internal memory but no number
It is called LRU according to block (memory block), operating system can belong to LRU according to which data and be moved out internal memory
And vacating space loads other data.
Expired processing is carried out by the data by frequency of use less than preset value in local cache, can be eliminated
The low data of frequency of use so that through operation after a while, hot spot data can in local substantial amounts of caching,
The access cached to remote distributed can be reduced.
S13：If the data are not all stored in local cache and remote distributed caching, from data
Read the data in storehouse.
Application server in local cache and remote distributed caching it is determined that all do not store called side needs
Data when, the read request for carrying data id can be sent to database, so that read from database should
The corresponding data of data id, obtain the data that called side needs.
S14：After the data are read, the data are write into local cache and remote distributed is cached
In.
Application server from database after the data are read, in the way of L2 cache, are locally being delayed
Deposit and remote distributed caching in all store the data.
In the present embodiment, cached by writing data into local cache and remote distributed, it is possible to achieve two grades
Caching, so as to first can be read when needing to read data in local cache, it is to avoid focus on long-distance distribution
Read in formula caching, it is to avoid the request of the back end server of some focus occurs waiting or time-out, it is to avoid
Influence is brought on service availability, it is ensured that business trouble-free operation.Further, by right in local cache
The relatively low data of frequency of use carry out expired processing, and hot spot data can be accumulated in local cache, reduce
The access cached to remote distributed.
When above-described embodiment is given in local cache and remote distributed caching all in the absence of the data needed
Handling process, when local cache or remote distributed caching in exist need data when, it can be performed
His flow.Referring specifically to following embodiments.
Fig. 2 is the schematic flow sheet of the date storage method of the application another embodiment proposition.Fig. 3 is figure
The schematic diagram of 2 corresponding system interactions.
Referring to Fig. 2, this method includes：
S201：Application server, which is received, carries data id in read request, the read request.
Wherein, read request is the request for being used to read data that called side is sent.
Virtual memory (Virtual Memory, VM), VM can be disposed referring to Fig. 3, on application server
Include processing module and cache module, cache module includes local cache and remote distributed cache client
(referred to as cache client), remote distributed caching system includes multiple back end servers
(datanode server)。
Specifically, the processing module that the read request that called side is sent is application server is received.
Afterwards, processing module is received after read request, and the data carried in read request are inquired about to cache module
The corresponding data of id, if storing the corresponding data of data id in cache module, cache module should
Data return to processing module, and return to called side by processing module.If do not stored in cache module
The corresponding data of data id, then cache module is empty message to processing module returning result, handles mould
Block is received after the message, and the read request for carrying data id is sent to database, is obtained from database
The data simultaneously return to called side.
Wherein, when any of local cache and remote distributed caching store data id corresponding data,
The result that cache module returns to processing module includes the corresponding data of data id, local cache and long-range
When the corresponding data of data id are not all stored in distributed caching, cache module returning result is empty
Message.
Specifically, referring to Fig. 2, this method also includes：
S202：Application server inquires about the data id for whether storing and being carried in read request in local cache,
If so, performing S203, otherwise, S204 is performed.
Wherein, local cache can correspond to the data of data storage id and particular content in data storage,
The data id for whether storing and being carried in read request is inquired about in local cache so as to elder generation.
S203：Application server obtains the corresponding data of data id in local cache, and returns to and call
Side.
Wherein, the data of data storage id and particular content can be corresponded in local cache, therefore, according to
The corresponding relation, can obtain the corresponding data of data id carried in read request.
Specifically, map data structures can be used when data cached in local cache.
Wherein, map is a kind of data structure, refers to store number in the form of key-value pair (key-value)
According to.In the present embodiment, key can be specifically data id, value be particular content data.
Expiration policies in local cache can use lru algorithm or use Guava Cache mechanism, from
Expiration policies are defined, the data to frequency of use less than preset value carry out expired processing.
Wherein, during LRU memory management a kind of page replacement algorithm, in internal memory but no number
It is called LRU according to block (memory block), operating system can belong to LRU according to which data and be moved out internal memory
And vacating space loads other data.
Guava Cache are that a kind of local cache is realized, support a variety of caching expiration policies.
Expired processing is carried out by the data by frequency of use less than preset value in local cache, can be eliminated
The low data of frequency of use so that through operation after a while, hot spot data can in local substantial amounts of caching,
The access cached to remote distributed can be reduced.
S204：Application server inquires about the data for whether storing and being carried in read request in remote distributed caching
Id, if so, performing S205, otherwise, performs S206.
Specifically, as shown in figure 3, cache client is used to calculate routing rule (route rule), really
Determine the back end server that data id belongs to carried in read request, and in corresponding back end service
Inquired about in device and whether store the corresponding data of data id.Wherein, each back end server can be deposited
Store up the corresponding data of data id of preset range.
Processing module is received after the carrying data id of called side transmission read request, is forwarded to cache module
The read request, cache module is received after the read request, and first inquiry whether there is the data in local cache
Id, if it does not, the cache client in cache module calculates routing rule, determines that data id belongs to
In back end server, such as, data id belongs to the first scope, and the data of the first scope are
First back end server storage, then the back end server that data id belongs to is the first data
Node server.So that the data id back end servers belonged to are the first back end servers as an example,
Afterwards, the read request for carrying data id can be sent to the first back end server by cache client,
Whether the first back end server can be inquired about itself stores data id, to be carried out according to Query Result
Respective handling.
S205：Application server obtains the corresponding data of data id in remote distributed caching, by this
In data write-in local cache, and return to called side.
, can be according to the data id of storage for example, stored in the first back end server after data id
Corresponding relation between data, obtains the corresponding data of data id, and be sent in application server
Cache client.
Cache client is received after the data of the first back end server transmission, can be write the data into
In local cache.And, the data are returned to processing module by local cache, and tune is returned to by processing module
With side.
S206：Application server obtains the corresponding data of data id in database.
If for example, in local cache and remote distributed caching during the corresponding data of all not data id,
Cache module can be empty message to processing module returning result.
Processing module is received after the message that the result is sky, and the reading for carrying data id is sent to database
Request, database can obtain data id according to the corresponding relation between the data id and data of storage
Corresponding data simultaneously return to the processing module in application server.
S207：Application server is write the data into local cache and remote distributed caching.
For example, processing module is received after the data of database return, cache module is written into.Cache mould
On the one hand block writes the data into local cache in processing, on the other hand, road is calculated by cache client
By rule, corresponding back end server in remote distributed caching system is determined, and write the data into
Corresponding back end server.
In the present embodiment, by the data that needs are not all stored in local cache and remote distributed caching
When, data are read from database, and write local cache and remote distributed caching, it is possible to achieve two grades
Caching, when reading data, first to be read from local cache, it is to avoid the request of data will be read
Focus in far-field distribution formula caching, it is ensured that business trouble-free operation.In addition, being deposited when in remote distributed caching
When containing the data not having in local cache, data can be obtained from remote distributed caching and are write local
In caching, data are subsequently obtained from local cache to facilitate.In addition, to frequency of use in local cache
Data less than preset value carry out expired processing, and hot spot data can be caused in local substantial amounts of caching, reduction
The access cached to remote distributed.
, can be using such as when occurring data renewal after data storage, it may appear that data more news
Under mode fail local cache and remote distributed caching in data.
Fig. 4 be the application one embodiment propose Stale Cache in data schematic flow sheet.Fig. 5
It is the corresponding system interaction schematic diagrames of Fig. 4.
Referring to Fig. 4, the method for the present embodiment includes：
S401：Application server updates the data the data stored in storehouse after data update.
Wherein, data update can be by application (APP) initiate, some APP data renewal after,
Data after renewal can be sent to the processing module of application server, processing module is received after renewal
After data, corresponding data in storehouse are updated the data.
For example, the first data id data there occurs renewal (being updated to A ' by A), then application server
Write request can be sent to database, write request includes the data A ' after the first data id and renewal,
Database is received after the write request, and the corresponding data of the first data id are updated into A ' from A.
After data update, on the one hand application server can update the data the data in storehouse, the opposing party
Face, can also be carried out at failure to storing the data updated in local cache and remote distributed caching
Reason.The flow of specific crash handling can include：
S402：Application server sends the variation notice that data update to message server.
Wherein, the data id updated can be carried by changing in notifying, such as the first data update,
The data id of the first data can then be carried, it is assumed that represented with the first data id.
S403：Message server sends multicast message to all application servers.
Wherein, message server is received after above-mentioned variation notifies, with multicast mode to all application services
Device sends the message for indicating data failure, with the corresponding data that fail.
The data id changed, such as the first data id are carried in the multicast message that message server is sent,
With first data that fail.
S404：Application server is received after the multicast message, the number updated in failure local cache
According to.
Wherein, each application server is received after multicast message, can parse data from the message
Id, and fail in local cache data id and corresponding data.
S405：The data updated in application server failure remote distributed caching.
Wherein, each application server is received after multicast message, the respective counts in the local cache that not only fails
According to, in addition it is also necessary to the data in being cached to far-field distribution formula carry out crash handling.
Specifically, the flow of the data in failure remote distributed caching can include：
It is determined that store the data remote distributed caching in back end server；
The message for the mark for carrying the data is sent to the back end server, so that the data section
Point server is received after the message, the data according to the indicating failure.
For example, processing module is received after multicast message, sent to cache module and carry the first data id's
Failure instruction, cache module is received after failure instruction, and on the one hand fail the first data in local cache
Id and corresponding first data, on the other hand, the first data are determined by the cache client in cache module
The back end server (assuming that being the first back end server) that id belongs to, then cache client is to
One back end server sends the failure instruction for carrying the first data id, and the first back end server connects
Receive after failure instruction, the first data id and corresponding of failure the first back end server memory storage
First data.
The present embodiment with each application server fail remote distributed caching in data instance, Ke Yili
Solution, can also determine an application server according to preset algorithm in all application servers,
The failure of data in remote distributed caching is carried out by an application server, and specifically determines some and answers
Do not limited with the algorithm of server.
Further, after the data in local cache and remote distributed caching all fail, if subsequently needed
The data are read, due to all failing, then this can not be read from local cache and remote distributed caching
Data, called side needs to read data from database afterwards, and writes local cache and remote distributed is slow
In depositing, thus realize local cache and remote distributed caching in data renewal, and with the number in database
According to being consistent.
In the present embodiment, by occur data update when, failure local cache and remote distributed caching in
Corresponding data, data can be read from database again in the subsequent read data, and will again
In the data write-in local cache and remote distributed caching of reading, local cache and remote distributed are realized
Data in caching update, and are consistent with the data in database.In addition, the present embodiment is in failure number
According to when, by message server using multicast mode carry out message push, treatment effeciency can be improved.
Fig. 6 is the structural representation for the data storage device that the application one embodiment is proposed.
The device of the present embodiment can be located in application server.The place in above-described embodiment can specifically be corresponded to
Manage module.
Referring to Fig. 6, the device 60 includes：Receiving module 61, enquiry module 62, the and of read module 63
Writing module 64.
Receiving module 61 is used to receive the read request for being used for reading data.
For example, when called side needs to read data, can be sent out to application server (APP server)
Send read request.
Enquiry module 62 is corresponding for inquiring about the read request in local cache and remote distributed caching
Data.
Wherein, data can correspond to the mark of data storage and the data of particular content, read request in storage
In can carry the mark (data id) of data, so as in the data of storage inquiry with the presence or absence of read
The mark carried in request, and, when storing data id according to above-mentioned corresponding relation, acquisition pair
The data answered.
Optionally, enquiry module 62 specifically for：
The corresponding data of the read request are first inquired about in local cache；
If the data are not present in local cache, the data are inquired about in remote distributed caching.
For example, called side can carry data id, using clothes when needing to read data in read request
Business device is received after read request, can first be inquired about in local cache and whether be stored data id, if do not had
There is storage, then inquiry can whether there is data id in remote distributed caching, if also do not deposited
Storage, it may be determined that local cache and remote distributed caching all do not store the data that called side needs to read.
If read module 63 is used to all not store the number in local cache and remote distributed caching
According to then reading the data from database.
Application server in local cache and remote distributed caching it is determined that all do not store called side needs
Data when, the read request for carrying data id can be sent to database, so that read from database should
The corresponding data of data id, obtain the data that called side needs.
Writing module 64 is used for after the data are read, and the data are write into local cache and long-range
In distributed caching.
Application server from database after the data are read, in the way of L2 cache, are locally being delayed
Deposit and remote distributed caching in all store the data.
In addition, in some embodiments, local cache is used for：If there is the number in remote distributed caching
According to obtaining from remote distributed caching and store the data.
In some embodiments, referring to Fig. 7, the device 60 also includes：
Update module 65, for after the data update, updating the data the number stored in storehouse
According to.
Wherein, data update can be by application (APP) initiate, some APP data renewal after,
Data after renewal can be sent to the processing module of application server, processing module is received after renewal
After data, corresponding data in storehouse are updated the data.
For example, the first data id data there occurs renewal (being updated to A ' by A), then application server
Write request can be sent to database, write request includes the data A ' after the first data id and renewal,
Database is received after the write request, and the corresponding data of the first data id are updated into A ' from A.
Failed module 66, for being carried out to the data stored in local cache and remote distributed caching
Crash handling.
After data update, on the one hand application server can update the data the data in storehouse, the opposing party
Face, can also be carried out at failure to storing the data updated in local cache and remote distributed caching
Reason.
Further, referring to Fig. 7, failed module 66 includes：
First module 661, for sending the variation notice that data update to message server, so that described disappear
Breath server is received after the variation notice, and multicast message is sent to all application servers.
Wherein, the data id updated can be carried by changing in notifying, such as the first data update,
The data id of the first data can then be carried, it is assumed that represented with the first data id.
Wherein, message server is received after above-mentioned variation notifies, with multicast mode to all application services
Device sends the message for indicating data failure, with the corresponding data that fail.
The data id changed, such as the first data id are carried in the multicast message that message server is sent,
With first data that fail.
Second unit 662, for receiving the number stored in the local cache that failed after the multicast message
According to.
Wherein, each application server is received after multicast message, can parse data from the message
Id, and fail in local cache data id and corresponding data.
Third unit 663, for the data stored in the remote distributed caching that fails.
Wherein, each application server is received after multicast message, the respective counts in the local cache that not only fails
According to, in addition it is also necessary to the data in being cached to far-field distribution formula carry out crash handling.
Optionally, the third unit 663 specifically for：
It is determined that store the data remote distributed caching in back end server；
The message for the mark for carrying the data is sent to the back end server, so that the data section
Point server is received after the message, the data according to the indicating failure.
For example, processing module is received after multicast message, sent to cache module and carry the first data id's
Failure instruction, cache module is received after failure instruction, and on the one hand fail the first data in local cache
Id and corresponding first data, on the other hand, the first data are determined by the cache client in cache module
The back end server (assuming that being the first back end server) that id belongs to, then cache client is to
One back end server sends the failure instruction for carrying the first data id, and the first back end server connects
Receive after failure instruction, the first data id and corresponding of failure the first back end server memory storage
First data.
Further, after the data in local cache and remote distributed caching all fail, if subsequently needed
The data are read, due to all failing, then this can not be read from local cache and remote distributed caching
Data, called side needs to read data from database afterwards, and writes local cache and remote distributed is slow
In depositing, thus realize local cache and remote distributed caching in data renewal, and with the number in database
According to being consistent.
Further, local cache can be right using lru algorithm or using guava cache mechanism
Frequency of use carries out expired processing less than the data of preset value.
It is understood that the present embodiment is corresponding with above method embodiment, particular content may refer to method
Embodiment, is no longer described in detail herein.
In the present embodiment, cached by writing data into local cache and remote distributed, it is possible to achieve two grades
Caching, so as to first can be read when needing to read data in local cache, it is to avoid focus on long-distance distribution
Read in formula caching, it is to avoid the request of the back end server of some focus occurs waiting or time-out, it is to avoid
Influence is brought on service availability, it is ensured that business trouble-free operation.
Fig. 8 is the structural representation for the data-storage system that the application one embodiment is proposed.Referring to Fig. 8,
The system 80 includes data storage device 81, local cache 82 and remote distributed caching 83.
Wherein, data storage device can as is seen in fig. 6 or fig. 7, local cache and remote distributed caching
For data storage.
Further, the data that local cache can be to frequency of use less than preset value carry out expired processing, with
Hot spot data is accumulated in local cache, the access cached to remote distributed is reduced.
In the present embodiment, cached by writing data into local cache and remote distributed, it is possible to achieve two grades
Caching, so as to first can be read when needing to read data in local cache, it is to avoid focus on long-distance distribution
Read in formula caching, it is to avoid the request of the back end server of some focus occurs waiting or time-out, it is to avoid
Influence is brought on service availability, it is ensured that business trouble-free operation.
It should be noted that in the description of the present application, term " first ", " second " etc. are only used for retouching
Purpose is stated, and it is not intended that indicating or implying relative importance.In addition, in the description of the present application, removing
Non- to be otherwise noted, the implication of " multiple " refers at least two.
Any process described otherwise above or method description are construed as in flow chart or herein, table
Show including it is one or more be used for realize specific logical function or process the step of executable instruction generation
Module, fragment or the part of code, and the scope of the preferred embodiment of the application includes other realization,
Wherein can not by order that is shown or discussing, including according to involved function by it is basic and meanwhile in the way of
Or in the opposite order, carrying out perform function, this should be by embodiments herein those of skill in the art
Member is understood.
It should be appreciated that each several part of the application can be realized with hardware, software, firmware or combinations thereof.
In the above-described embodiment, multiple steps or method can in memory and by suitable instruction be held with storage
The software or firmware that row system is performed are realized.If for example, realized with hardware, and in another embodiment party
It is the same in formula, it can be realized with any one of following technology well known in the art or their combination：Have
For the discrete logic for the logic gates that logic function is realized to data-signal, with suitable combination
The application specific integrated circuit of logic gates, programmable gate array (PGA), field programmable gate array (FPGA)
Deng.
Those skilled in the art be appreciated that to realize the whole that above-described embodiment method carries or
Part steps can be by program to instruct the hardware of correlation to complete, and described program can be stored in one kind
In computer-readable recording medium, the program upon execution, including one of the step of embodiment of the method or its group
Close.
In addition, each functional unit in the application each embodiment can be integrated in a processing module,
Can also be that unit is individually physically present, can also two or more units be integrated in a module
In.Above-mentioned integrated module can both be realized in the form of hardware, it would however also be possible to employ software function module
Form is realized.If the integrated module is realized using in the form of software function module and is used as independent product
Sale in use, can also be stored in a computer read/write memory medium.
Storage medium mentioned above can be read-only storage, disk or CD etc..
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ",
The description of " specific example " or " some examples " etc. means to combine that the embodiment or example describe is specific
Feature, structure, material or feature are contained at least one embodiment of the application or example.In this theory
In bright book, identical embodiment or example are not necessarily referring to the schematic representation of above-mentioned term.Moreover,
Specific features, structure, material or the feature of description can be in any one or more embodiments or examples
In combine in an appropriate manner.
Although embodiments herein has been shown and described above, it is to be understood that above-described embodiment
It is exemplary, it is impossible to be interpreted as the limitation to the application, one of ordinary skill in the art is the application's
In the range of above-described embodiment can be changed, change, replace and modification.
Claims (10)
1. a kind of date storage method, it is characterised in that including：
Receive the read request for reading data；
The corresponding data of the read request are inquired about in local cache and remote distributed caching；
If all not storing the data in local cache and remote distributed caching, read from database
Take the data；
After the data are read, the data are write in local cache and remote distributed caching.
2. according to the method described in claim 1, it is characterised in that the local cache is to frequency of use
Data less than preset value carry out expired processing.
3. according to the method described in claim 1, it is characterised in that also include：
After the data update, the data stored in storehouse are updated the data；
Crash handling is carried out to the data stored in local cache and remote distributed caching.
4. method according to claim 3, it is characterised in that described to local cache and long-range point
The data stored in cloth caching carry out crash handling, including：
To message server the variation that data update is sent to notify so that the message server receive it is described
Change after notifying, multicast message is sent to all application servers, so that application server receives described group
Broadcast what is stored in the data stored in the local cache that failed after message, and failure remote distributed caching
The data.
5. method according to claim 4, it is characterised in that the failure remote distributed caching
The data of middle storage, including：
It is determined that store the data remote distributed caching in back end server；
The message for the mark for carrying the data is sent to the back end server, so that the data section
Point server is received after the message, the data according to the indicating failure.
6. according to the method described in claim 1, it is characterised in that described in local cache and long-range point
The corresponding data of the read request are inquired about in cloth caching, including：
The corresponding data of the read request are first inquired about in local cache；
If the data are not present in local cache, the data are inquired about in remote distributed caching.
7. method according to claim 6, it is characterised in that also include：
If there are the data in remote distributed caching, the number is obtained from remote distributed caching
According to；
The data are write in local cache.
8. a kind of data storage device, it is characterised in that including：
Receiving module, for receiving the read request for being used for reading data；
Enquiry module, for inquiring about the corresponding number of the read request in local cache and remote distributed caching
According to；
Read module, if for all not storing the data in local cache and remote distributed caching,
The data are then read from database；
Writing module, for after the data are read, the data to be write into local cache and long-range point
In cloth caching.
9. device according to claim 8, it is characterised in that also include：
Update module, for after the data update, updating the data the data stored in storehouse；
Failed module, for being failed to the data stored in local cache and remote distributed caching
Processing.
10. a kind of data-storage system, it is characterised in that including：
Device as described in claim any one of 8-9；And,
Local cache and remote distributed caching；
The local cache and remote distributed are cached for data storage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610178110.6A CN107231395A (en) | 2016-03-25 | 2016-03-25 | Date storage method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610178110.6A CN107231395A (en) | 2016-03-25 | 2016-03-25 | Date storage method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107231395A true CN107231395A (en) | 2017-10-03 |
Family
ID=59932818
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610178110.6A Pending CN107231395A (en) | 2016-03-25 | 2016-03-25 | Date storage method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107231395A (en) |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108777718A (en) * | 2018-06-22 | 2018-11-09 | 阿里巴巴集团控股有限公司 | A kind of operation system passes through client and accesses reads the method and apparatus for writing few system more |
| CN109388656A (en) * | 2018-09-04 | 2019-02-26 | 中国建设银行股份有限公司 | Data processing method and system, device and storage medium based on multi-level buffer |
| CN109669960A (en) * | 2018-12-25 | 2019-04-23 | 钛马信息网络技术有限公司 | The system and method for caching snowslide is avoided by multi-level buffer in micro services |
| CN109818900A (en) * | 2017-11-20 | 2019-05-28 | 高德软件有限公司 | A kind of data management system and application server |
| CN109885593A (en) * | 2019-02-21 | 2019-06-14 | 北京字节跳动网络技术有限公司 | Method and apparatus for handling information |
| CN110046029A (en) * | 2019-03-13 | 2019-07-23 | 平安科技（深圳）有限公司 | Data processing method and device applied to multi-level buffer in cluster |
| CN110069419A (en) * | 2018-09-04 | 2019-07-30 | 中国平安人寿保险股份有限公司 | Multilevel cache system and its access control method, equipment and storage medium |
| CN110362426A (en) * | 2019-06-21 | 2019-10-22 | 华中科技大学 | A kind of selective copy realization method and system towards sudden load |
| CN110472167A (en) * | 2019-06-03 | 2019-11-19 | 北京京东振世信息技术有限公司 | Data managing method, device and computer readable storage medium |
| CN110895539A (en) * | 2019-11-22 | 2020-03-20 | 南威软件股份有限公司 | Second-level caching method of application system |
| CN111026761A (en) * | 2019-12-11 | 2020-04-17 | 上海鲸骞金融信息服务有限公司 | Financial data storage system, processing method and device |
| CN111405019A (en) * | 2020-03-10 | 2020-07-10 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN111464615A (en) * | 2020-03-30 | 2020-07-28 | 北京达佳互联信息技术有限公司 | Request processing method, device, server and storage medium |
| CN111694865A (en) * | 2020-06-02 | 2020-09-22 | 中国工商银行股份有限公司 | Four-layer structure data acquisition method and device based on distributed system |
| CN111737298A (en) * | 2020-06-19 | 2020-10-02 | 中国工商银行股份有限公司 | Cache data control method and device based on distributed storage |
| CN111897847A (en) * | 2020-08-07 | 2020-11-06 | 上海莉莉丝科技股份有限公司 | Data reading and writing method, system, device and medium based on multi-level cache |
| CN111988427A (en) * | 2020-08-31 | 2020-11-24 | 湖南快乐阳光互动娱乐传媒有限公司 | Distributed local cache processing method and system |
| CN112100537A (en) * | 2020-07-28 | 2020-12-18 | 苏宁云计算有限公司 | Data acquisition method and application server based on local cache and distributed cache |
| CN112328632A (en) * | 2020-10-09 | 2021-02-05 | 福建亿榕信息技术有限公司 | Distributed two-level caching method and system |
| CN112445859A (en) * | 2019-08-30 | 2021-03-05 | 北京数聚鑫云信息技术有限公司 | Data management method, device and system |
| CN112487111A (en) * | 2020-12-16 | 2021-03-12 | 江苏苏宁云计算有限公司 | Data table association method and device based on KV database |
| CN112559560A (en) * | 2019-09-10 | 2021-03-26 | 北京京东振世信息技术有限公司 | Metadata reading method and device, metadata updating method and device, and storage device |
| CN113031864A (en) * | 2021-03-19 | 2021-06-25 | 上海众源网络有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN115913646A (en) * | 2022-10-21 | 2023-04-04 | 网易（杭州）网络有限公司 | Method and device for intercepting blacklist object, electronic equipment and storage medium |
| CN115905306A (en) * | 2022-12-26 | 2023-04-04 | 北京滴普科技有限公司 | Local caching method, equipment and medium for OLAP analysis database |
| CN111737298B (en) * | 2020-06-19 | 2024-04-26 | 中国工商银行股份有限公司 | Cache data management and control method and device based on distributed storage |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521252A (en) * | 2011-11-17 | 2012-06-27 | 四川长虹电器股份有限公司 | Access method of remote data |
| CN103246696A (en) * | 2013-03-21 | 2013-08-14 | 宁波公众信息产业有限公司 | High-concurrency database access method and method applied to multi-server system |
| CN104142896A (en) * | 2013-05-10 | 2014-11-12 | 阿里巴巴集团控股有限公司 | Cache control method and system |
| CN104866434A (en) * | 2015-06-01 | 2015-08-26 | 北京圆通慧达管理软件开发有限公司 | Multi-application-oriented data storage system and data storage and calling method |
| CN105183394A (en) * | 2015-09-21 | 2015-12-23 | 北京奇虎科技有限公司 | Data storage processing method and device |
-
2016
- 2016-03-25 CN CN201610178110.6A patent/CN107231395A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102521252A (en) * | 2011-11-17 | 2012-06-27 | 四川长虹电器股份有限公司 | Access method of remote data |
| CN103246696A (en) * | 2013-03-21 | 2013-08-14 | 宁波公众信息产业有限公司 | High-concurrency database access method and method applied to multi-server system |
| CN104142896A (en) * | 2013-05-10 | 2014-11-12 | 阿里巴巴集团控股有限公司 | Cache control method and system |
| CN104866434A (en) * | 2015-06-01 | 2015-08-26 | 北京圆通慧达管理软件开发有限公司 | Multi-application-oriented data storage system and data storage and calling method |
| CN105183394A (en) * | 2015-09-21 | 2015-12-23 | 北京奇虎科技有限公司 | Data storage processing method and device |
Cited By (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109818900A (en) * | 2017-11-20 | 2019-05-28 | 高德软件有限公司 | A kind of data management system and application server |
| CN109818900B (en) * | 2017-11-20 | 2021-11-26 | 阿里巴巴（中国）有限公司 | Data management system and application server |
| CN108777718A (en) * | 2018-06-22 | 2018-11-09 | 阿里巴巴集团控股有限公司 | A kind of operation system passes through client and accesses reads the method and apparatus for writing few system more |
| CN109388656A (en) * | 2018-09-04 | 2019-02-26 | 中国建设银行股份有限公司 | Data processing method and system, device and storage medium based on multi-level buffer |
| CN110069419A (en) * | 2018-09-04 | 2019-07-30 | 中国平安人寿保险股份有限公司 | Multilevel cache system and its access control method, equipment and storage medium |
| CN109669960A (en) * | 2018-12-25 | 2019-04-23 | 钛马信息网络技术有限公司 | The system and method for caching snowslide is avoided by multi-level buffer in micro services |
| CN109885593A (en) * | 2019-02-21 | 2019-06-14 | 北京字节跳动网络技术有限公司 | Method and apparatus for handling information |
| CN110046029A (en) * | 2019-03-13 | 2019-07-23 | 平安科技（深圳）有限公司 | Data processing method and device applied to multi-level buffer in cluster |
| CN110472167A (en) * | 2019-06-03 | 2019-11-19 | 北京京东振世信息技术有限公司 | Data managing method, device and computer readable storage medium |
| CN110472167B (en) * | 2019-06-03 | 2022-11-08 | 北京京东振世信息技术有限公司 | Data management method, device and computer readable storage medium |
| CN110362426A (en) * | 2019-06-21 | 2019-10-22 | 华中科技大学 | A kind of selective copy realization method and system towards sudden load |
| CN112445859A (en) * | 2019-08-30 | 2021-03-05 | 北京数聚鑫云信息技术有限公司 | Data management method, device and system |
| CN112559560A (en) * | 2019-09-10 | 2021-03-26 | 北京京东振世信息技术有限公司 | Metadata reading method and device, metadata updating method and device, and storage device |
| CN110895539A (en) * | 2019-11-22 | 2020-03-20 | 南威软件股份有限公司 | Second-level caching method of application system |
| CN111026761A (en) * | 2019-12-11 | 2020-04-17 | 上海鲸骞金融信息服务有限公司 | Financial data storage system, processing method and device |
| CN111026761B (en) * | 2019-12-11 | 2024-04-02 | 上海鲸骞金融信息服务有限公司 | Financial data storage system, processing method and device |
| CN111405019A (en) * | 2020-03-10 | 2020-07-10 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN111405019B (en) * | 2020-03-10 | 2021-11-16 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN111464615A (en) * | 2020-03-30 | 2020-07-28 | 北京达佳互联信息技术有限公司 | Request processing method, device, server and storage medium |
| CN111464615B (en) * | 2020-03-30 | 2023-06-20 | 北京达佳互联信息技术有限公司 | Request processing method, device, server and storage medium |
| CN111694865A (en) * | 2020-06-02 | 2020-09-22 | 中国工商银行股份有限公司 | Four-layer structure data acquisition method and device based on distributed system |
| CN111737298B (en) * | 2020-06-19 | 2024-04-26 | 中国工商银行股份有限公司 | Cache data management and control method and device based on distributed storage |
| CN111737298A (en) * | 2020-06-19 | 2020-10-02 | 中国工商银行股份有限公司 | Cache data control method and device based on distributed storage |
| CN112100537A (en) * | 2020-07-28 | 2020-12-18 | 苏宁云计算有限公司 | Data acquisition method and application server based on local cache and distributed cache |
| WO2022022105A1 (en) * | 2020-07-28 | 2022-02-03 | 苏宁易购集团股份有限公司 | Data acquisition method based on local cache and distributed cache, and application server |
| CN111897847A (en) * | 2020-08-07 | 2020-11-06 | 上海莉莉丝科技股份有限公司 | Data reading and writing method, system, device and medium based on multi-level cache |
| CN111988427A (en) * | 2020-08-31 | 2020-11-24 | 湖南快乐阳光互动娱乐传媒有限公司 | Distributed local cache processing method and system |
| CN112328632A (en) * | 2020-10-09 | 2021-02-05 | 福建亿榕信息技术有限公司 | Distributed two-level caching method and system |
| CN112328632B (en) * | 2020-10-09 | 2022-06-14 | 福建亿榕信息技术有限公司 | Distributed two-level caching method and system |
| CN112487111A (en) * | 2020-12-16 | 2021-03-12 | 江苏苏宁云计算有限公司 | Data table association method and device based on KV database |
| CN113031864B (en) * | 2021-03-19 | 2024-02-02 | 上海众源网络有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN113031864A (en) * | 2021-03-19 | 2021-06-25 | 上海众源网络有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN115913646A (en) * | 2022-10-21 | 2023-04-04 | 网易（杭州）网络有限公司 | Method and device for intercepting blacklist object, electronic equipment and storage medium |
| CN115905306A (en) * | 2022-12-26 | 2023-04-04 | 北京滴普科技有限公司 | Local caching method, equipment and medium for OLAP analysis database |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107231395A (en) | Date storage method, device and system | |
| CN107528816B (en) | Processing method, management system and server of ID in distributed database | |
| US8949533B2 (en) | Method and node entity for enhancing content delivery network | |
| EP2555129B1 (en) | Method and system to maintain strong consistency of distributed replicated contents in a client/server system | |
| US10887419B2 (en) | Fast cache purge optimization handling of unavailable nodes | |
| CN107533438B (en) | Data replication in a memory system | |
| CN107888657A (en) | Low latency distributed memory system | |
| CN105635196B (en) | A kind of method, system and application server obtaining file data | |
| CN101751415B (en) | Metadata service system, metadata synchronized method and writing server updating method | |
| CN107025289B (en) | A kind of method and relevant device of data processing | |
| EP2710477B1 (en) | Distributed caching and cache analysis | |
| CN101706817B (en) | Distributed file system and garbage data cleaning method thereof | |
| US10326854B2 (en) | Method and apparatus for data caching in a communications network | |
| WO2001084338A2 (en) | Cluster configuration repository | |
| KR20080014800A (en) | Efficient processing of time-bounded messages | |
| CN110069419A (en) | Multilevel cache system and its access control method, equipment and storage medium | |
| CN113268472B (en) | Distributed data storage system and method | |
| CN102438020A (en) | Method and equipment for distributing contents in content distribution network, and network system | |
| CN103581052A (en) | Data processing method, router and NDN system | |
| CN102281332A (en) | Distributed cache array and data updating method thereof | |
| CN101404649B (en) | Data processing system based on CACHE and its method | |
| CN108616581A (en) | Data-storage system and method based on OLAP/OLTP mixing applications | |
| CN106940696B (en) | Information query method and system for SDN multi-layer controller | |
| CN107992270B (en) | Method and device for globally sharing cache of multi-control storage system | |
| US20120297010A1 (en) | Distributed Caching and Cache Analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200925 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20200925 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171003 |