CN110046029A - Data processing method and device applied to multi-level buffer in cluster - Google Patents
Data processing method and device applied to multi-level buffer in cluster Download PDFInfo
- Publication number
- CN110046029A CN110046029A CN201910190657.1A CN201910190657A CN110046029A CN 110046029 A CN110046029 A CN 110046029A CN 201910190657 A CN201910190657 A CN 201910190657A CN 110046029 A CN110046029 A CN 110046029A
- Authority
- CN
- China
- Prior art keywords
- host
- business datum
- local cache
- read
- cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/466—Transaction processing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of data processing method and device applied to multi-level buffer in cluster.Cluster includes the first host with the first local cache and the second host with the second local cache, this method comprises: when the first host can not read business datum from the first local cache, business datum is read from network buffer system, when can not read business datum from network buffer system, it accesses database and reads business datum, and after access database reads business datum, business datum is written in the first local cache and network buffer system；After network buffer system is written into business datum, to the message of the second host publication write operation；When second host listens to the message of write operation, business datum is written to the second local cache.Through the invention, it rationally realizes the multi-level buffer in data buffer storage, is able to ascend issued transaction speed.
Description
Technical field
The present invention relates to multi-level buffer technical field more particularly to a kind of data processings applied to multi-level buffer in cluster
Method, apparatus, computer equipment and storage medium.
Background technique
For financial system or life trading place systems distributed system, the cluster being made of multiple main frames is generally included,
Each host in cluster can access identical database as an application node.Local cache has been locally located in each host,
Host obtains data to local cache first, when the data failure in local cache can not be normal during issued transaction
When acquisition, then to database acquisition, under normal circumstances, the data of each host local cache, due to data age etc., often
There is mass data frequency at this point, each host requires to reacquire data to database respectively in the situation that will appear while failing
The phenomenon that numerous access database, reduce the processing speed of affairs.
Therefore it provides a kind of applied to the data processing method of multi-level buffer, device, computer equipment and storage in cluster
Medium promotes the speed of issued transaction rationally to realize the multi-level buffer in data buffer storage, is this field technology urgently to be solved
Problem.
Summary of the invention
The object of the present invention is to provide a kind of applied to the data processing method of multi-level buffer, device, computer in cluster
Equipment and storage medium, for solving the above problem of the existing technology.
To achieve the goals above, the present invention provides a kind of data processing methods applied to multi-level buffer in cluster.
This is applied in cluster in the data processing method of multi-level buffer, and cluster includes the first host and the second host, the
The local cache of one host is the first local cache, and the local cache of the second host is the second local cache, data processing method
It include: that the first host executes the instruction that required business datum is read from the first local cache；When the first host can not be from
When reading business datum in one local cache, the first host executes the instruction that business datum is read from network buffer system；
When the first host can not read business datum from network buffer system, the first host accesses database to read business number
According to；The first host access database read business datum after, the first host by business datum be written the first local cache and
In network buffer system；After network buffer system is written into business datum, to the message of the second host publication write operation；Second is main
Machine monitors the message of network buffer system publication；When the second host listens to the message of write operation, business datum is written to
Second local cache.
To achieve the goals above, the present invention provides a kind of data processing equipments applied to multi-level buffer in cluster.
Cluster includes the first host and the second host, and the local cache of the first host is the first local cache, the second host
Local cache be the second local cache, the first host and the second host include the data for being applied to multi-level buffer in cluster
Processing unit, data processing equipment include the first read module, the second read module, third read module, the first writing module,
Monitor module and the second writing module, in which: the first read module of the first host from the first local cache for reading institute
The business datum needed；Second read module of the first host is used to not reading from the first local cache when the first read module
When to business datum, business datum is read from network buffer system；The third read module of first host is used to read when second
When modulus block can not read business datum from network buffer system, access database reads business datum；First host
First writing module is used for after third read module access database reads business datum, and first is written in business datum
In ground caching and network buffer system；The monitoring module of second host monitors the message of network buffer system publication for machine,
In, after network buffer system is written into business datum, issue the message of write operation；Second writing module of the second host is for working as
When listening to the message of write operation, business datum is written to the second local cache.
To achieve the goals above, the present invention provides a kind of computer equipments.
The computer equipment include memory, processor and storage on a memory and the meter that can run on a processor
Calculation machine program, processor execute program when realize it is provided by the invention any one be applied to cluster in multi-level buffer data at
The step of reason method.
To achieve the goals above, the present invention provides a kind of computer readable storage mediums.
The computer readable storage medium is stored thereon with computer program, this hair is realized when program is executed by processor
Any one of bright offer is applied to the step of data processing method of multi-level buffer in cluster.
Compared with prior art, using provided by the invention applied to the data processing method of multi-level buffer, dress in cluster
It sets, computer equipment and storage medium, at least the following technical effects are achieved:
Host is obtained when carrying out transaction operation according to the successive priority of local cache, network buffer system and database
Business datum is taken, the speed of transaction operation execution is able to ascend；Meanwhile after a host gets business datum from database,
The data for updating own local caching, itself pass through the probability that local cache gets business datum to increase next time, also,
Business datum is diffused into the local of other hosts in cluster by network buffer system by the data for updating network buffer system
In caching, increase the probability that other hosts get business datum by local cache, to promote host in cluster on the whole
Handle the speed of transaction operation.
Detailed description of the invention
Fig. 1 is the flow chart provided in an embodiment of the present invention applied to the data processing method of multi-level buffer in cluster；
Fig. 2 is the block diagram provided in an embodiment of the present invention applied to the data processing equipment of multi-level buffer in cluster；
Fig. 3 is the schematic diagram of computer equipment provided in an embodiment of the present invention.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that described herein, specific examples are only used to explain the present invention, not
For limiting the present invention.Based on the embodiments of the present invention, those of ordinary skill in the art are not before making creative work
Every other embodiment obtained is put, shall fall within the protection scope of the present invention.
Data processing method, device, computer equipment and storage provided by the invention applied to multi-level buffer in cluster
Medium, cluster include multiple main frames, and each host is able to carry out the operation in distributed transaction, are locally respectively provided with this in each host
Ground caching, carries out the storage of business datum, in the implementation procedure of transaction operation, obtains data, Neng Gouti by local cache
Local cache is considered as level cache to promote the completion speed of whole affairs by the speed that lift operations execute.
Meanwhile network buffer system is set and is used as L2 cache, any host can work as local cache and can not obtain in cluster
When getting data, access network buffer system obtains data, and when can not also obtain data by network buffer system, then go
It accesses database and obtains data, also, after accessing database, update local cache and network buffer system.
It is important to, after network buffer system updates, gives out information to other hosts in cluster, each host can
The message issued on network buffer system is monitored, therefore, after network buffer system gives out information, host, which can listen to, to disappear
The data of network buffer system and are diffused to respective local cache by breath, update its local cache, thus, a host from
After data are accessed in database, the All hosts to cluster may be updated, to promote host process thing in cluster on the whole
The speed of business operation.
Hereinafter the number provided by the invention applied to multi-level buffer in cluster will be further described by specifically embodiment
According to processing method, device, computer equipment and storage medium.
Fig. 1 is the flow chart provided in an embodiment of the present invention applied to the data processing method of multi-level buffer in cluster, should
Embodiment provides a kind of data processing method applied to multi-level buffer in cluster, wherein cluster includes multiple main frames, the reality
It applies in example and is illustrated by taking the first host and the second host in cluster as an example, the local cache for defining the first host is first
Ground caching, define the second host local cache be the second local cache for, as shown in Figure 1, the embodiment provide application
Include the following steps, namely S1 to step S7 in the data processing method of multi-level buffer in cluster.
The S1: the first host of step executes the instruction that required business datum is read from the first local cache.
In order to promote the speed that operation executes, some business datums are cached in local cache, therefore, in transaction operation
In implementation procedure, the business datum that the first host needs is obtained from the first local cache first, still, the data of local cache
With certain timeliness, when the data failure of local cache, the first host can not read industry from the first local cache
Business data.
When the first host can not read business datum from the first local cache, following step S2 is executed, when first
When host can read business datum from the first local cache, transaction operation is executed using the business datum read.
The S2: the first host of step executes the instruction that business datum is read from network buffer system.
Local cache as level cache, network buffer system as L2 cache, when read in level cache less than
When business datum, it is read out by L2 cache.
When the first host can not read business datum from network buffer system, following step S3 is executed, when first
When host can read business datum from network buffer system, transaction operation is executed using the business datum read, together
When, the business datum read is stored in the first local cache.
Wherein, Redis storage system can be used in network buffer system, and cache memory can be used in local cache.
Redis storage system provides the interface of access Redis storage system to each host, and step S2 is specifically included:
First host calls the interface of access Redis storage system, reads business datum from Redis storage system.
The S3: the first host of step accesses database to read business datum.
It when also reading in L2 cache less than business datum, needs to access database, at this point, the first host passes through visit
It asks database, executes transaction operation using the business datum read from database.In addition, accessing database in the first host
After reading business datum, following step S4 is continued to execute.
Business datum is written in the first local cache and network buffer system the S4: the first host of step.
First host will also be read other than executing transaction operation using the business datum read from database
Business datum be written in the first local cache and network buffer system, can be from the thus when carrying out reading data next time
Business datum is read in one local cache, it, can be from network-caching system when the business datum in the first local cache fails
Business datum is read in system.
Step S5: after network buffer system is written into business datum, to the message of the second host publication write operation.
For network buffer system, after network buffer system is written into business datum, the host publication into cluster is write
The message of operation, optionally, network buffer system can be by way of broadcast, and the All hosts into cluster issue write operation
Message；Alternatively, optionally, network buffer system can remove the first host into cluster and (business datum be written to network buffer system
Host) except other hosts publication write operation message, in this embodiment, the host in cluster in addition to the first host
Including the second host.
The S6: the second host of step monitors the message of network buffer system publication.
Each host, which all has, monitors the function that gives out information of network buffer system, thus, network buffer system is to second
After host issues the message of write operation, the second host can listen to the message, also, when the second host listens to write operation
When message, following step S7 is executed.
Business datum is written to the second local cache by the S7: the second host of step.
Using the data processing method provided by the invention applied to multi-level buffer in cluster, host is carrying out transaction operation
When, business datum is obtained according to the successive priority of local cache, network buffer system and database, is able to ascend transaction operation
The speed of execution；Meanwhile after a host gets business datum from database, the data of own local caching are updated, to increase
Add and itself pass through the probability that local cache gets business datum next time, also, update the data of network buffer system, passes through
Business datum is diffused into cluster in the local cache of other hosts by network buffer system, is increased other hosts and is passed through local
Caching gets the probability of business datum, to promote the speed of host process transaction operation in cluster on the whole.
In one embodiment, the data processing method applied to multi-level buffer in cluster include thes steps that as follows: the
The value of business datum in one the first local cache of host modifications and database, the first host is by business number modified in database
According to being synchronized in network buffer system.
Specifically, the first host needs to modify the value of business datum during executing transaction operation, such as occurs one
Secondary purchase needs to modify in inventory database the value of remaining inventory, at this point, in modification database the value of business datum it is same
When, the value of business datum in the first local cache is also modified, the value of business datum in the first local cache can be made to keep newest
State, in subsequent transaction operation, the value of the business datum got by the first local cache is latest value；Also, the
One host by synchronizing traffic data modified in database into network buffer system so that business number in network buffer system
According to value keep newest state, in subsequent transaction operation, no matter the first host or the second host, can pass through network
The value for the business datum that caching system is got is latest value.
It in one embodiment, include: first the step of the value of business datum in first the first local cache of host modifications
Host obtains the version information of business datum in the first local cache, obtains first version information；It is slow that first host obtains network
The version information of business datum in deposit system, obtains second edition information；First host judges first version information and the second edition
Whether this information is identical；When first version information is identical as second edition information, in first the first local cache of host modifications
The value of business datum；When first version information and second edition information difference, the first host first will be in network buffer system
Then synchronizing traffic data modifies the value of business datum in the first local cache to the first local cache again.
The data processing method for being applied to multi-level buffer in cluster provided using the embodiment, when the second host is to data
Business datum value in library is modified, and by synchronizing traffic data modified in database into network buffer system with
Afterwards, the value for the business datum that the first host is got from the first local cache may lag current value, in order to avoid
Initial value in one the first local cache of host modifications when business datum lags current value, and the version information that data are arranged carries out pipe
Reason, that is, in first the first local cache of host modifications when the value of business datum, first to business datum in the first local cache
Version information and network buffer system in the version information of business datum be compared, if inconsistent, illustrate that network is slow
Business datum in deposit system may be by other host modifications, and business datum in the first local cache of the first host
Value is not modified current value, therefore, now by the synchronizing traffic data in network buffer system to the first local cache,
The value for guaranteeing the first local cache is current value, is then modified again to the value of the business datum in the first local cache.Example
Such as during panic buying, single purchase occurs on each host, can modify the value of remaining inventory in inventory database, and will remain
The value of remaining inventory is synchronized to network buffer system, and each host needs to first pass through remaining inventory's version in the value of modification local cache
The value of the remaining inventory of this information assurance local cache is the current value of remaining inventory.
The above are the data processing methods provided by the invention applied to multi-level buffer in cluster, and the present invention also provides one
Kind is applied to the data processing equipment of multi-level buffer in cluster.It should be noted that provided by the invention applied to more in cluster
The data processing equipment of grade caching with it is above-mentioned to be applied to the data processing method of multi-level buffer in cluster corresponding, can in place of related
Mutually reference.
Fig. 2 is the block diagram provided in an embodiment of the present invention applied to the data processing equipment of multi-level buffer in cluster, such as Fig. 2
Shown, this embodiment offers a kind of data processing equipments applied to multi-level buffer in cluster, wherein cluster includes several masters
Machine, includes the data processing equipment in each host, in the embodiment by taking cluster includes the first host and the second host as an example
It is illustrated, wherein the local cache of the first host is the first local cache, and the local cache of the second host is second local slow
It deposits, the first host and the second host include data processing equipment, as shown in Fig. 2, the data processing equipment includes the first reading
Module 10, third read module 30, the first writing module 40, monitors module 50 and the second writing module at second read module 20
60, in which:
First read module 10 of the first host from the first local cache for reading required business datum；First is main
Second read module 20 of machine is used for when the first read module 10 can not read business datum from the first local cache, from
Business datum is read in network buffer system；The third read module 30 of first host is used for when the second read module 20 can not be from
When reading business datum in network buffer system, access database reads business datum；First writing module of the first host
40 in third read module 30 for accessing after database reads business datum, by business datum be written the first local cache and
In network buffer system；The monitoring module 50 of second host monitors the message of network buffer system publication for machine, wherein network
After caching system is written into business datum, the message of write operation is issued；Second writing module 60 of the second host is used for when monitoring
To write operation message when, business datum is written to the second local cache.
Meanwhile second host the first read module 10 for reading required business datum from the second local cache；
Second read module 20 of the second host is used to that business number can not to be read from the second local cache when the first read module 10
According to when, business datum is read from network buffer system；The third read module 30 of second host is used for when the second read module
20 from network buffer system when can not read business datum, and access database reads business datum；The first of second host
Writing module 40 is used for after the access database of third read module 30 reads business datum, and business datum is written second
In ground caching and network buffer system；The monitoring module 50 of first host monitors the message of network buffer system publication for machine,
Wherein, after network buffer system is written into business datum, the message of write operation is issued；Second writing module 60 of the first host is used
In when listening to the message of write operation, business datum is written to the first local cache.
Optionally, data processing equipment further includes modified module and synchronization module, wherein the modified module of the first host is used
In the value for modifying business datum in the first local cache and database；The synchronization module of first host in database for will modify
Synchronizing traffic data afterwards is into network buffer system.The modified module of second host is for modifying the second local cache and data
The value of business datum in library；The synchronization module of second host is for delaying synchronizing traffic data modified in database to network
In deposit system.
Optionally, modified module include: first acquisition unit, second acquisition unit, judging unit, first modification unit and
Second modification unit, wherein the acquiring unit of the first host is used to obtain the version information of business datum in the first local cache,
Obtain first version information；The second acquisition unit of first host is used to obtain the version letter of business datum in network buffer system
Breath, obtains second edition information；The judging unit of first host is for judging whether are first version information and second edition information
It is identical；First modification unit of the first host is used for when first version information is identical as second edition information, modifies first
The value of business datum in ground caching；Second modification unit of the first host is used to work as first version information and second edition information not
Meanwhile first then the synchronizing traffic data in network buffer system to the first local cache being modified in the first local cache again
The value of business datum.Meanwhile second the acquiring unit of host be used to obtain the version information of business datum in the second local cache,
Obtain first version information；The second acquisition unit of second host is used to obtain the version letter of business datum in network buffer system
Breath, obtains second edition information；The judging unit of second host is for judging whether are first version information and second edition information
It is identical；First modification unit of the second host is used for when first version information is identical as second edition information, modifies second
The value of business datum in ground caching；Second modification unit of the second host is used to work as first version information and second edition information not
Meanwhile first then the synchronizing traffic data in network buffer system to the second local cache being modified in the second local cache again
The value of business datum.
Optionally, network buffer system is Redis storage system, and local cache is cache memory.
Optionally, the second read module 20 calls access Redis to deposit when reading business datum from network buffer system
The interface of storage system reads business datum from Redis storage system.
The embodiment of the present invention also provides a kind of computer equipment, can such as execute the smart phone, tablet computer, pen of program
Remember that this computer, desktop computer, rack-mount server, blade server, tower server or Cabinet-type server are (including only
Server cluster composed by vertical server or multiple servers) etc..The computer equipment 02 of the present embodiment includes at least
But it is not limited to: memory 021, the processor 022 of connection can be in communication with each other by system bus, as shown in Figure 3.It may be noted that
It is that Fig. 3 illustrates only the computer equipment 02 with component 021-022, it should be understood that being not required for implementing all show
Component out, the implementation that can be substituted is more or less component.
In the present embodiment, memory 021 (i.e. readable storage medium storing program for executing) includes flash memory, hard disk, multimedia card, card-type memory
(for example, SD or DX memory etc.), random access storage device (RAM), static random-access memory (SRAM), read-only memory
(ROM), electrically erasable programmable read-only memory (EEPROM), programmable read only memory (PROM), magnetic storage, magnetic
Disk, CD etc..In some embodiments, memory 021 can be the internal storage unit of computer equipment 02, such as the calculating
The hard disk or memory of machine equipment 02.In further embodiments, memory 021 is also possible to the external storage of computer equipment 02
The plug-in type hard disk being equipped in equipment, such as the computer equipment 02, intelligent memory card (Smart Media Card, SMC), peace
Digital (Secure Digital, SD) card, flash card (Flash Card) etc..Certainly, memory 21 can also both include meter
The internal storage unit for calculating machine equipment 02 also includes its External memory equipment.In the present embodiment, memory 021 is commonly used in storage
It is installed on the operating system and types of applications software of computer equipment 02, such as the multistage in cluster that is applied to of above-described embodiment is delayed
The program code etc. for the data processing equipment deposited.It has exported or has incited somebody to action in addition, memory 021 can be also used for temporarily storing
The Various types of data to be exported.
Processor 022 can be in some embodiments central processing unit (Central Processing Unit, CPU),
Controller, microcontroller, microprocessor or other data processing chips.The processor 022 is commonly used in control computer equipment
02 overall operation.In the present embodiment, program code or processing number of the processor 022 for being stored in run memory 021
According to, such as data processing equipment etc. of the operation applied to multi-level buffer in cluster.
The embodiment of the present invention also provides a kind of computer readable storage medium, as flash memory, hard disk, multimedia card, card-type are deposited
Reservoir (for example, SD or DX memory etc.), static random-access memory (SRAM), read-only is deposited random access storage device (RAM)
Reservoir (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read only memory (PROM), magnetic storage,
Disk, CD, server, App are stored thereon with computer program, realization when program is executed by processor using store etc.
Corresponding function.The computer readable storage medium of the present embodiment is used to store the data processing dress for being applied to multi-level buffer in cluster
It sets, the data processing method for being applied to multi-level buffer in cluster of above-described embodiment is realized when being executed by processor.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of data processing method applied to multi-level buffer in cluster, which is characterized in that the cluster includes the first host
With the second host, the local cache of first host is the first local cache, and the local cache of second host is second
Local cache, the data processing method include:
First host executes the instruction that required business datum is read from first local cache；
When first host can not read the business datum from first local cache, first host is held
Row reads the instruction of the business datum from network buffer system；
When first host can not read the business datum from the network buffer system, first host is visited
Database is asked to read the business datum；
It is accessed after the database reads the business datum in first host, first host is by the business number
According in write-in first local cache and the network buffer system；
After the network buffer system is written into the business datum, the second host of Xiang Suoshu issues the message of write operation；
Second host monitors the message of the network buffer system publication；
When second host listens to the message of the write operation, it is local slow that the business datum is written to described second
It deposits.
2. the data processing method according to claim 1 applied to multi-level buffer in cluster, which is characterized in that the number
According to processing method further include:
The value of business datum described in first local cache described in first host modifications and the database；
First host is by the synchronizing traffic data modified in the database into the network buffer system.
3. the data processing method according to claim 2 applied to multi-level buffer in cluster, which is characterized in that described the
The step of value of business datum described in first local cache described in one host modifications includes:
First host obtains the version information of business datum described in first local cache, obtains first version letter
Breath；
First host obtains the version information of business datum described in the network buffer system, obtains second edition letter
Breath；
First host judges whether the first version information and the second edition information are identical；
When the first version information is identical as the second edition information, first is local slow described in first host modifications
The value of business datum described in depositing；
When the first version information and the second edition information difference, first host is first by the network-caching system
Then the synchronizing traffic data in system modifies business described in first local cache to first local cache again
The value of data.
4. the data processing method according to claim 1 applied to multi-level buffer in cluster, which is characterized in that the net
Network caching system is Redis storage system, and the local cache is cache memory.
5. the data processing method according to claim 4 applied to multi-level buffer in cluster, which is characterized in that described the
The one host execution the step of instruction of the business datum is read from network buffer system includes:
First host calls the interface for accessing the Redis storage system, from the Redis storage system described in reading
Business datum.
6. a kind of data processing equipment applied to multi-level buffer in cluster, which is characterized in that the cluster includes the first host
With the second host, the local cache of first host is the first local cache, and the local cache of second host is second
Local cache, first host and second host include the data processing equipment, the data processing equipment packet
It includes the first read module, the second read module, third read module, the first writing module, monitor module and the second writing module,
Wherein:
First read module of first host is for reading required business datum from first local cache；
Second read module of first host is used for when first read module can not be from first local cache
When reading the business datum, the business datum is read from network buffer system；
The third read module of first host is used for when second read module can not be from the network buffer system
When reading the business datum, access database reads the business datum；
First writing module of first host is used to read in the third read module access database described
After business datum, the business datum is written in first local cache and the network buffer system；
The monitoring module of second host monitors the message of the network buffer system publication for machine, wherein the network
After caching system is written into the business datum, the message of write operation is issued；
Second writing module of second host is used for when listening to the message of the write operation, and the business datum is write
Enter to second local cache.
7. the data processing equipment according to claim 6 applied to multi-level buffer in cluster, which is characterized in that the number
According to processing unit further include: modified module and synchronization module, in which:
The modified module of first host is for modifying business datum described in first local cache and the database
Value；
The synchronization module of first host is used for the synchronizing traffic data modified in the database to the net
In network caching system.
8. the data processing equipment according to claim 7 applied to multi-level buffer in cluster, which is characterized in that
The modified module include: first acquisition unit, second acquisition unit, judging unit, first modification unit and second repair
Change unit, in which:
The acquiring unit of first host is used to obtain the version information of business datum described in first local cache, obtains
To first version information；
The second acquisition unit of first host is used to obtain the version letter of business datum described in the network buffer system
Breath, obtains second edition information；
The judging unit of first host is for judging whether the first version information and the second edition information are identical；
First modification unit of first host is used for when the first version information is identical as the second edition information,
Modify the value of business datum described in first local cache；
Second modification unit of first host is used for when the first version information and the second edition information difference,
First then the synchronizing traffic data in the network buffer system is modified described first to first local cache again
The value of business datum described in local cache.
9. a kind of computer equipment, can run on a memory and on a processor including memory, processor and storage
Computer program, which is characterized in that the processor realizes any one of claim 1 to 5 the method when executing described program
The step of.
10. a kind of computer readable storage medium, is stored thereon with computer program, it is characterised in that: described program is processed
The step of any one of claim 1 to 5 the method is realized when device executes.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910190657.1A CN110046029A (en) | 2019-03-13 | 2019-03-13 | Data processing method and device applied to multi-level buffer in cluster |
| PCT/CN2019/117410 WO2020181810A1 (en) | 2019-03-13 | 2019-11-12 | Data processing method and apparatus applied to multi-level caching in cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910190657.1A CN110046029A (en) | 2019-03-13 | 2019-03-13 | Data processing method and device applied to multi-level buffer in cluster |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110046029A true CN110046029A (en) | 2019-07-23 |
Family
ID=67273726
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910190657.1A Pending CN110046029A (en) | 2019-03-13 | 2019-03-13 | Data processing method and device applied to multi-level buffer in cluster |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110046029A (en) |
| WO (1) | WO2020181810A1 (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110765165A (en) * | 2019-10-23 | 2020-02-07 | 中国银行股份有限公司 | Method, device and system for synchronously processing cross-system data |
| CN111143382A (en) * | 2019-12-13 | 2020-05-12 | 新华三大数据技术有限公司 | Data processing method, system and computer readable storage medium |
| WO2020181810A1 (en) * | 2019-03-13 | 2020-09-17 | 平安科技（深圳）有限公司 | Data processing method and apparatus applied to multi-level caching in cluster |
| CN111708288A (en) * | 2020-05-18 | 2020-09-25 | 慧灵科技（深圳）有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN111782352A (en) * | 2019-11-29 | 2020-10-16 | 北京沃东天骏信息技术有限公司 | Service scheduling method and device |
| CN111966283A (en) * | 2020-07-06 | 2020-11-20 | 云知声智能科技股份有限公司 | Client multi-level caching method and system based on enterprise-level super-computation scene |
| CN112445859A (en) * | 2019-08-30 | 2021-03-05 | 北京数聚鑫云信息技术有限公司 | Data management method, device and system |
| CN112632092A (en) * | 2020-12-18 | 2021-04-09 | 北京浪潮数据技术有限公司 | Cluster management method, device, equipment and storage medium |
| CN113360425A (en) * | 2021-06-28 | 2021-09-07 | 深圳市高德信通信股份有限公司 | Distributed multi-level cache system |
| CN115190125A (en) * | 2022-06-27 | 2022-10-14 | 京东科技控股股份有限公司 | Monitoring method and device for cache cluster |
| CN117851456A (en) * | 2024-01-05 | 2024-04-09 | 迪爱斯信息技术股份有限公司 | Method, system and server for sharing data in cluster |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106790629A (en) * | 2017-01-03 | 2017-05-31 | 努比亚技术有限公司 | Data synchronization unit and its realize the method for data syn-chronization, client access system |
| CN107231395A (en) * | 2016-03-25 | 2017-10-03 | 阿里巴巴集团控股有限公司 | Date storage method, device and system |
| CN109388656A (en) * | 2018-09-04 | 2019-02-26 | 中国建设银行股份有限公司 | Data processing method and system, device and storage medium based on multi-level buffer |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9201892B2 (en) * | 2011-08-30 | 2015-12-01 | International Business Machines Corporation | Fast snapshots |
| CN107463511B (en) * | 2017-01-23 | 2020-06-26 | 北京思特奇信息技术股份有限公司 | Data internationalization realization method and device based on multi-level cache |
| CN110046029A (en) * | 2019-03-13 | 2019-07-23 | 平安科技（深圳）有限公司 | Data processing method and device applied to multi-level buffer in cluster |
-
2019
- 2019-03-13 CN CN201910190657.1A patent/CN110046029A/en active Pending
- 2019-11-12 WO PCT/CN2019/117410 patent/WO2020181810A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107231395A (en) * | 2016-03-25 | 2017-10-03 | 阿里巴巴集团控股有限公司 | Date storage method, device and system |
| CN106790629A (en) * | 2017-01-03 | 2017-05-31 | 努比亚技术有限公司 | Data synchronization unit and its realize the method for data syn-chronization, client access system |
| CN109388656A (en) * | 2018-09-04 | 2019-02-26 | 中国建设银行股份有限公司 | Data processing method and system, device and storage medium based on multi-level buffer |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020181810A1 (en) * | 2019-03-13 | 2020-09-17 | 平安科技（深圳）有限公司 | Data processing method and apparatus applied to multi-level caching in cluster |
| CN112445859A (en) * | 2019-08-30 | 2021-03-05 | 北京数聚鑫云信息技术有限公司 | Data management method, device and system |
| CN110765165B (en) * | 2019-10-23 | 2022-07-29 | 中国银行股份有限公司 | Method, device and system for synchronously processing cross-system data |
| CN110765165A (en) * | 2019-10-23 | 2020-02-07 | 中国银行股份有限公司 | Method, device and system for synchronously processing cross-system data |
| CN111782352A (en) * | 2019-11-29 | 2020-10-16 | 北京沃东天骏信息技术有限公司 | Service scheduling method and device |
| CN111143382A (en) * | 2019-12-13 | 2020-05-12 | 新华三大数据技术有限公司 | Data processing method, system and computer readable storage medium |
| CN111143382B (en) * | 2019-12-13 | 2022-08-09 | 新华三大数据技术有限公司 | Data processing method, system and computer readable storage medium |
| CN111708288A (en) * | 2020-05-18 | 2020-09-25 | 慧灵科技（深圳）有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN111708288B (en) * | 2020-05-18 | 2021-11-26 | 慧灵科技（深圳）有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN111966283A (en) * | 2020-07-06 | 2020-11-20 | 云知声智能科技股份有限公司 | Client multi-level caching method and system based on enterprise-level super-computation scene |
| CN112632092A (en) * | 2020-12-18 | 2021-04-09 | 北京浪潮数据技术有限公司 | Cluster management method, device, equipment and storage medium |
| CN113360425A (en) * | 2021-06-28 | 2021-09-07 | 深圳市高德信通信股份有限公司 | Distributed multi-level cache system |
| CN115190125A (en) * | 2022-06-27 | 2022-10-14 | 京东科技控股股份有限公司 | Monitoring method and device for cache cluster |
| CN117851456A (en) * | 2024-01-05 | 2024-04-09 | 迪爱斯信息技术股份有限公司 | Method, system and server for sharing data in cluster |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020181810A1 (en) | 2020-09-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110046029A (en) | Data processing method and device applied to multi-level buffer in cluster | |
| CN108234641B (en) | Data reading and writing method and device based on distributed consistency protocol | |
| CN106899648A (en) | A kind of data processing method and equipment | |
| CN111723079A (en) | Data migration method and device, computer equipment and storage medium | |
| CN111880956B (en) | Data synchronization method and device | |
| CN110138688A (en) | Dynamic adjusts method, apparatus, equipment and the readable storage medium storing program for executing of business interface | |
| CN109885612B (en) | Synchronous validation method and device for intelligent contracts of block chains | |
| CN104065636A (en) | Data processing method and system | |
| CN111538519A (en) | Version upgrading method and device | |
| WO2020238860A1 (en) | Distributed file batch processing method and apparatus, and readable storage medium | |
| CN115269544A (en) | Database cluster upgrading method and device, electronic equipment and storage medium | |
| CN110737504A (en) | deep learning model training fault-tolerant method, system, terminal and storage medium | |
| CN103309696B (en) | Method and device for updating Java card extended library, and Java card | |
| CN110297822A (en) | Key management method, device, equipment and storage medium towards block chain | |
| CN110908644B (en) | Configuration method and device of state node, computer equipment and storage medium | |
| CN112015819A (en) | Data updating method, device, equipment and medium for distributed graph database | |
| CN111309746A (en) | Asynchronous parallel data synchronization method and device | |
| CN109241110A (en) | Order management method and system, electronic equipment, storage medium | |
| CN109257256A (en) | Apparatus monitoring method, device, computer equipment and storage medium | |
| CN105760283A (en) | Log output method and device | |
| CN114064713A (en) | Method, device and equipment for improving stability of dolphinscheduler platform and readable medium | |
| CN113590273A (en) | Transaction processing method, system, device and storage medium | |
| CN111752911A (en) | Data transmission method, system, terminal and storage medium based on Flume | |
| CN111858487A (en) | Data updating method and device | |
| CN112860746B (en) | Cache reduction-based method, equipment and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |