CN102333108A - Distributed cache synchronization system and method - Google Patents
Distributed cache synchronization system and method Download PDFInfo
- Publication number
- CN102333108A CN102333108A CN201110066128A CN201110066128A CN102333108A CN 102333108 A CN102333108 A CN 102333108A CN 201110066128 A CN201110066128 A CN 201110066128A CN 201110066128 A CN201110066128 A CN 201110066128A CN 102333108 A CN102333108 A CN 102333108A
- Authority
- CN
- China
- Prior art keywords
- data
- servers
- buffer memory
- server
- packet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention relates to a distributed system cache synchronization system, which comprises a plurality of servers, wherein each server comprises a cache unit used for caching needed data in a database, a monitoring unit used for monitoring requests of other servers used as receiving terminals or operations of the servers used as transmitting terminals, a conversion unit used for converting cache data changed in the server into packets or converting the packets into changed data, and a transmission unit used for transmitting the packets to other servers. The invention also relates to a distributed system coach synchronization method. The database access times are reduced, the speed for reading the cache is much higher than that for reading files and the database, and the reading speed is greatly raised. Distributed support can make the use of cache in a large high concurrent system be possible.
Description
Technical field
The present invention relates to a kind of distributed caching synchro system and method.
Background technology
Along with the fast development of telecommunication service, for the IT system of telecommunications industry company brings immense pressure.Data read efficiency and just highlight.System responses is slow, system is unavailable like BOSS (Business & Operation Support System, business operation support system), to client's satisfaction and the problem in using can't in time be known and improvement etc.
So developing can high-efficient disposition, improve the speed of visit data, the method that reduces system consumption also just become a kind of must needs.
Summary of the invention
To the efficiency that above-described data read, the present invention provides a kind of distributed caching synchro system and method.
The technical scheme that the present invention solves the problems of the technologies described above is following: the synchronous method of a kind of distributed system buffer memory, and it may further comprise the steps:
Step 1: desired data in every station server database is carried out buffer memory;
Step 2: when a station server is data cached when changing, the data with change in this server are assembled into packet;
Step 3: this server is given other servers with packet delivery;
Step 4: other servers receive said packet, packet are changed into the data of said change again;
Step 5: other servers carry out buffer memory with the data of said change.
Further, step 1 comprises following substep:
Step 1.1: the territory that on every station server, is configured in internal memory, store, promptly will carry out the data in buffer database data and be divided into value object by demand, add class name with the bag name of value object and form the unique territory in the buffer memory;
Step 1.2: the record in the database is read, and it is configured to entity class, and the entity class of the corresponding one-tenth of each bar record in the database;
Step 1.3: entity class is deposited in the territory with the key-value pair mode of Map, and wherein Key is a major key, and the Value value is corresponding entity class.
Further, every station server has disposed Listener (Listener has been that (Servlet is a kind of java application of server end to Servlet, has the characteristic that is independent of platform and agreement, can generate the dynamic Web page in the step 2.) monitor, it can monitor the request of client, the operation of service end etc.), when a station server is data cached when changing, Listener simultaneously dispense request to other servers.
Further, server becomes the data conversion of change through reflex mechanism the packet of Map structure in the step 2.
Further, step 3 is passed through the xmlrpc mode for this server, gives other servers with packet delivery.
A kind of distributed system buffer memory synchro system, it comprises multiple servers, said every station server comprises:
Buffer unit carries out buffer memory with desired data in the database;
Monitoring unit is monitored as the request of other servers of receiving terminal or as the operation of the server of transmitting terminal;
Conversion unit with the data cached packet that changes into of change in the server, or changes into packet the data of change;
Transmission unit, with said data packet transmission to other servers.
Further, it also comprises the emergent cache interface that loads, and when certain station server during when machine, said emergent loading cache interface reloads data cached, guarantees simultaneously and the proper communication of other servers.
The invention has the beneficial effects as follows:
1. reduce the database access times, one time reading database deposits buffer memory in, can accept to visit many times, accepts database resource.
2. the speed that reads buffer memory is much higher than and reads file and database, improves reading speed greatly.
3. distributed support can make large-scale high concurrent system adopt buffer memory to become possibility.
Description of drawings
Fig. 1 is the flow chart of distributed caching method for synchronous of the present invention;
Fig. 2 is the structured flowchart of distributed caching synchro system of the present invention.
Embodiment
Below in conjunction with accompanying drawing principle of the present invention and characteristic are described, institute gives an actual example and only is used to explain the present invention, is not to be used to limit scope of the present invention.
As shown in Figure 1, the synchronous method of a kind of distributed system buffer memory, it may further comprise the steps:
Step 1: desired data in every station server database is carried out buffer memory;
Step 2: when a station server is data cached when changing, the data with change in this server are assembled into packet;
Step 3: this server is given other servers with packet delivery;
Step 4: other servers receive said packet, packet are changed into the data of said change again;
Step 5: other servers carry out buffer memory with the data of said change.
Step 1.1: the territory that on every station server, is configured in internal memory, store, promptly will carry out the data in buffer database data and be divided into value object by demand, add class name with the bag name of value object and form the unique territory in the buffer memory;
Step 1.2: the record in the database is read, and it is configured to entity class, and the entity class of the corresponding one-tenth of each bar record in the database;
Step 1.3: entity class is deposited in the territory with the key-value pair mode of Map, and wherein Key is a major key, and the Value value is corresponding entity class.
As shown in Figure 2, a kind of distributed system buffer memory synchro system, it comprises multiple servers 1, said every station server 1 comprises:
It also comprises the emergent cache interface 5 that loads, and when certain station server during when machine, said emergent loading cache interface 5 reloads data cached, guarantees simultaneously and the proper communication of other servers.Can revise all data cached stationary problems.More can guarantee the accuracy of data.
Every station server 1 can be done transmitting terminal and receiving terminal.The security and stability of the system that this mode is just strengthened greatly is promptly when certain server occurs when problems such as machines.Do not influence the normal operation of other servers 1.
Adopt the program of java language specific as follows:
1. be to utilize the realization of reflex mechanism below with object (entity class) and the mutual commentaries on classics of Map:
/*
* with Map type of converting to
*/
public?static?void?map2Object(Object?obj,?Map?map)
throws?IllegalArgumentException,?IllegalAccessException?{
Field[]?fields?=?obj.getClass().getDeclaredFields();
for?(int?j?=?0;?j?<?fields.length;?j++)?{
Fields [j] .setAccessible (true); // this variable variable that scope check is provided with in class that do not conduct interviews is set private can be be
fields[j].set(obj,?map.get(fields[j].getName()));
}
}
/*
* convert class to Map
*/
public?static?void?object2Map(Object?obj,?Map?map)
throws?IllegalArgumentException,?IllegalAccessException?{
Field[]?fields?=?obj.getClass().getDeclaredFields();
for?(int?j?=?0;?j?<?fields.length;?j++)?{
Fields [j] .setAccessible (true); // this variable variable that scope check is provided with in class that do not conduct interviews is set private can be be
map.put(fields[j].getName(),?fields[j].get(obj));
}
}
/*
Obtain instance according to class name
*/
public?static?Object?getClassByName(String?className)?throws?Exception
{
Class?c;
Object?o;
c?=?Class.forName(className);
o?=?(c.getClassLoader().loadClass(className)).newInstance();
return?o;
}
2. be the realization that the transmitting terminal traversal is sent data below:
/*
The inlet traversal sends to each member in the cluster.
*/
public?boolean?distributData(List?list,String?className){
boolean?flag?=?true;
try{
=null && isDistribute.equals (" on ")) { // distributed caching switch.
convertDataforRpc(list,className);
for?(int?i?=?0;?i?<?host.length;?i++)?{
logger.info(host[i]?+?":"?+?port[i]?+?"/"?+?server[i]+?"?are?in?updata.....");
if?(!sendData(host[i],?port[i],?server[i],dataList))?{
logger.error("IndexServerError?error:"+?host[i]+?":"+?port[i]+?"/"+?server[i]+?"?faild?!");
flag?=?false;
}
}
}
}catch(Exception?e){
logger.error("error:?in?PromptDistributer.distributData()",e);
return?false;
}
return?flag;
}
It below 3 the realization of accepting data
/*
Accepting rpc passes the packet of coming and resolves synchronization caching sava again
*/
public?Boolean?receiveUpdateData(List?rpcData)?{
try?{
String?className?=?(String)rpcData.get(0);
logger.info("******seccess?received?bossDataDistrabutData***classname="+className);
Map?rpcDataMap?=?new?HashMap();
List?rpcRelultList?=?new?ArrayList();
// renewal buffer memory
SychronizedBossInfo?se?=?new?SychronizedBossInfo();
//**************************
Object?o?=?DistrabutTool.getClassByName(className);
For (int i=1; I < rpcData.size (); I++) />/ since 1.0 for being className
o?=?DistrabutTool.getClassByName(className);
rpcDataMap?=?(HashMap)?rpcData.get(i);
DistrabutTool.map2Object(o,rpcDataMap);
rpcRelultList.add(o);;
}
//**************************
If (className.equals (" com.sitech.kms.ehcache.pojo.UserInfo ")) { // user
se.updataUserByCach(rpcRelultList);
Else if (className.equals (" com.sitech.kms.ehcache.pojo.Dept ")) { // department
se.updataDeptByCach(rpcRelultList);
Else if (className.equals (" com.sitech.kms.ehcache.pojo.UserRole ")) { // user role
se.updataUserRoleByCach(rpcRelultList);
Else if (className.equals (" com.sitech.kms.ehcache.pojo.RoleMenu ")) { // role menu
se.updataRoleMenuByCach(rpcRelultList);
Else if (className.equals (" com.sitech.kms.ehcache.pojo.RoleModule ")) { // role function module
se.updataRoleModuleByCach(rpcRelultList);
}
}?catch?(Exception?e)?{
e.printStackTrace();
logger.error("IndexServerError?in?PromptDistributReceiver.receiveUpdateData",e);
return?new?Boolean("false")?;
}
return?new?Boolean("true");
}
In the industry during data of Query Database, all be directly to inquire about basically now with the sql statement.Each inquiry all can be to database build-up of pressure and resource consumption.Adopting behind the present invention can one time behind the reading database, unlimited accepting request.And what the present invention read is internal storage data, so speed is fast more a lot of than reading database.
Be applicable to fewer updating form data.Generally to use table (comprising update (legacy data in the updating form), insert (insertion), delete (deletion) etc.) in fewer execution write (data are write in the opened file) operation
The present invention has set forth through depositing numerous types of data in buffer memory.To improve the efficient to the data visit, one cutting system of efficiency when being applicable to data access.
The above is merely preferred embodiment of the present invention, and is in order to restriction the present invention, not all within spirit of the present invention and principle, any modification of being done, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (7)
1. method that the distributed system buffer memory is synchronous, it is characterized in that: it may further comprise the steps:
Step 1: desired data in every station server database is carried out buffer memory;
Step 2: when a station server is data cached when changing, the data with change in this server are assembled into packet;
Step 3: this server is given other servers with packet delivery;
Step 4: other servers receive said packet, packet are changed into the data of said change again;
Step 5: other servers carry out buffer memory with the data of said change.
2. the synchronous method of distributed system buffer memory according to claim 1, it is characterized in that: said step 1 comprises following substep:
Step 1.1: the territory that on every station server, is configured in internal memory, store, promptly will carry out the data in buffer database data and be divided into value object by demand, add class name with the bag name of value object and form the unique territory in the buffer memory;
Step 1.2: the record in the database is read, and it is configured to entity class, and the entity class of the corresponding one-tenth of each bar record in the database;
Step 1.3: entity class is deposited in the territory with the key-value pair mode of Map, and wherein Key is a major key, and the Value value is corresponding entity class.
3. the synchronous method of distributed system buffer memory according to claim 1 and 2; It is characterized in that: every station server has disposed Listener in the said step 2; When a station server is data cached when changing, Listener simultaneously dispense request to other servers.
4. the synchronous method of distributed system buffer memory according to claim 1 and 2 is characterized in that: server becomes the data conversion of change through reflex mechanism the packet of Map structure in the said step 2.
5. the synchronous method of distributed system buffer memory according to claim 1 and 2 is characterized in that: said step 3 is passed through the xmlrpc mode for this server, gives other servers with packet delivery.
6. distributed system buffer memory synchro system, it is characterized in that: it comprises multiple servers, said every station server comprises:
Buffer unit carries out buffer memory with desired data in the database;
Monitoring unit is monitored as the request of other servers of receiving terminal or as the operation of the server of transmitting terminal;
Conversion unit with the data cached packet that changes into of change in the server, or changes into packet the data of change;
Transmission unit, with said data packet transmission to other servers.
7. a kind of distributed system buffer memory synchro system according to claim 6; It is characterized in that: it also comprises the emergent cache interface that loads; When certain station server during when machine, said emergent loading cache interface reloads data cached, guarantees the proper communication with other servers simultaneously.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110066128A CN102333108A (en) | 2011-03-18 | 2011-03-18 | Distributed cache synchronization system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110066128A CN102333108A (en) | 2011-03-18 | 2011-03-18 | Distributed cache synchronization system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102333108A true CN102333108A (en) | 2012-01-25 |
Family
ID=45484711
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110066128A Pending CN102333108A (en) | 2011-03-18 | 2011-03-18 | Distributed cache synchronization system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102333108A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915338A (en) * | 2012-09-18 | 2013-02-06 | 浪潮(北京)电子信息产业有限公司 | Data synchronization method of distributed file systems and file systems |
| CN103116476A (en) * | 2013-02-22 | 2013-05-22 | 浪潮电子信息产业股份有限公司 | System for managing caches among multiple controllers |
| WO2014132148A1 (en) * | 2013-02-28 | 2014-09-04 | International Business Machines Corporation | Identifying and accessing reference data in an in-memory data grid |
| CN104202424A (en) * | 2014-09-19 | 2014-12-10 | 中国人民财产保险股份有限公司 | Method for extending cache by software architecture |
| CN104202423A (en) * | 2014-09-19 | 2014-12-10 | 中国人民财产保险股份有限公司 | System for extending caches by aid of software architectures |
| CN104468239A (en) * | 2014-12-22 | 2015-03-25 | 上海大唐移动通信设备有限公司 | Data processing method and device based on rules |
| CN104461929A (en) * | 2013-09-23 | 2015-03-25 | 中国银联股份有限公司 | Distributed type data caching method based on interceptor |
| CN106375362A (en) * | 2015-07-24 | 2017-02-01 | 广州市百果园信息技术有限公司 | Cache synchronization method and system for distributed server |
| CN106790562A (en) * | 2016-12-26 | 2017-05-31 | 中国建设银行股份有限公司 | A kind of distributed cache system, method and device |
| CN107153644A (en) * | 2016-03-02 | 2017-09-12 | 阿里巴巴集团控股有限公司 | A kind of method of data synchronization and device |
| CN103780636B (en) * | 2012-10-17 | 2018-02-27 | 阿里巴巴集团控股有限公司 | A kind of caching data processing method and system |
| CN112597197A (en) * | 2020-12-17 | 2021-04-02 | 杭州当虹科技股份有限公司 | Distributed cache system based on memory |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1754155A (en) * | 2003-01-17 | 2006-03-29 | 泰斯特网络公司 | Method and system for use of storage caching with a distributed file system |
| CN1851698A (en) * | 2005-09-30 | 2006-10-25 | 华为技术有限公司 | High frequency data access system and method under world-wide web interactive programming language platform environment |
| US20060271741A1 (en) * | 2005-05-24 | 2006-11-30 | International Business Machines Corporation | Data processing system, cache system and method for scrubbing a domain indication in response to execution of program code |
| CN101060402A (en) * | 2006-04-21 | 2007-10-24 | 盛趣信息技术（上海）有限公司 | Hash-map-based multipoint buffer memory device |
| CN101169790A (en) * | 2007-11-30 | 2008-04-30 | 中兴通讯股份有限公司 | Matrix type data caching method and device based on WEB application |
| US20090198899A1 (en) * | 2008-01-31 | 2009-08-06 | Bea Systems, Inc. | System and method for transactional cache |
-
2011
- 2011-03-18 CN CN201110066128A patent/CN102333108A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1754155A (en) * | 2003-01-17 | 2006-03-29 | 泰斯特网络公司 | Method and system for use of storage caching with a distributed file system |
| US20060271741A1 (en) * | 2005-05-24 | 2006-11-30 | International Business Machines Corporation | Data processing system, cache system and method for scrubbing a domain indication in response to execution of program code |
| CN1851698A (en) * | 2005-09-30 | 2006-10-25 | 华为技术有限公司 | High frequency data access system and method under world-wide web interactive programming language platform environment |
| CN101060402A (en) * | 2006-04-21 | 2007-10-24 | 盛趣信息技术（上海）有限公司 | Hash-map-based multipoint buffer memory device |
| CN101169790A (en) * | 2007-11-30 | 2008-04-30 | 中兴通讯股份有限公司 | Matrix type data caching method and device based on WEB application |
| US20090198899A1 (en) * | 2008-01-31 | 2009-08-06 | Bea Systems, Inc. | System and method for transactional cache |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915338A (en) * | 2012-09-18 | 2013-02-06 | 浪潮(北京)电子信息产业有限公司 | Data synchronization method of distributed file systems and file systems |
| CN103780636B (en) * | 2012-10-17 | 2018-02-27 | 阿里巴巴集团控股有限公司 | A kind of caching data processing method and system |
| CN103116476A (en) * | 2013-02-22 | 2013-05-22 | 浪潮电子信息产业股份有限公司 | System for managing caches among multiple controllers |
| CN103116476B (en) * | 2013-02-22 | 2016-12-28 | 浪潮电子信息产业股份有限公司 | A kind of cache management system between multi-controller |
| US9244630B2 (en) | 2013-02-28 | 2016-01-26 | International Business Machines Corporation | Identifying and accessing reference data in an in-memory data grid |
| WO2014132148A1 (en) * | 2013-02-28 | 2014-09-04 | International Business Machines Corporation | Identifying and accessing reference data in an in-memory data grid |
| US9229659B2 (en) | 2013-02-28 | 2016-01-05 | International Business Machines Corporation | Identifying and accessing reference data in an in-memory data grid |
| CN104461929B (en) * | 2013-09-23 | 2018-03-23 | 中国银联股份有限公司 | Distributed data cache method based on blocker |
| CN104461929A (en) * | 2013-09-23 | 2015-03-25 | 中国银联股份有限公司 | Distributed type data caching method based on interceptor |
| CN104202423B (en) * | 2014-09-19 | 2016-01-20 | 中国人民财产保险股份有限公司 | A kind of system by software architecture expansion buffer memory |
| CN104202424B (en) * | 2014-09-19 | 2016-01-27 | 中国人民财产保险股份有限公司 | A kind of method using software architecture to expand buffer memory |
| CN104202423A (en) * | 2014-09-19 | 2014-12-10 | 中国人民财产保险股份有限公司 | System for extending caches by aid of software architectures |
| CN104202424A (en) * | 2014-09-19 | 2014-12-10 | 中国人民财产保险股份有限公司 | Method for extending cache by software architecture |
| CN104468239A (en) * | 2014-12-22 | 2015-03-25 | 上海大唐移动通信设备有限公司 | Data processing method and device based on rules |
| CN104468239B (en) * | 2014-12-22 | 2018-08-21 | 上海大唐移动通信设备有限公司 | A kind of rule-based data processing method and device |
| CN106375362A (en) * | 2015-07-24 | 2017-02-01 | 广州市百果园信息技术有限公司 | Cache synchronization method and system for distributed server |
| CN107153644A (en) * | 2016-03-02 | 2017-09-12 | 阿里巴巴集团控股有限公司 | A kind of method of data synchronization and device |
| CN106790562A (en) * | 2016-12-26 | 2017-05-31 | 中国建设银行股份有限公司 | A kind of distributed cache system, method and device |
| CN112597197A (en) * | 2020-12-17 | 2021-04-02 | 杭州当虹科技股份有限公司 | Distributed cache system based on memory |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102333108A (en) | Distributed cache synchronization system and method | |
| CN101409614B (en) | Data synchronization method, system and equipment | |
| US9880982B2 (en) | System and method for rendering presentation pages based on locality | |
| US20130275958A1 (en) | Automatic identification of services | |
| USRE44836E1 (en) | System and method for efficient transfer of applications and data during device swap | |
| KR101863398B1 (en) | Method and system for synchronization mechanism on multi-server reservation system | |
| CN101847100B (en) | Method for expanding software application and device | |
| CN101266542B (en) | Method and device for loading component in container | |
| US20110016514A1 (en) | Methods for monitoring and control of electronic devices | |
| Sapkota et al. | Distributed web service discovery architecture | |
| US7650609B2 (en) | Multi-environment document management system access | |
| CN103209223A (en) | Distributed application conversation information sharing method and system and application server | |
| CN103473696A (en) | Method and system for collecting, analyzing and distributing internet business information | |
| US20070011274A1 (en) | Data transfer in a multi-environment document management system access | |
| EP2778968B1 (en) | Mobile telecommunication device remote access to cloud-based or virtualized database systems | |
| CN103853714A (en) | Data processing method and device | |
| US20060259523A1 (en) | System and method of synchronization of internal data cache with wireless device application data repositories | |
| CN103034738A (en) | Relevant database for managing heterogeneous unstructured data and method for creating and inquiring description information of unstructured data thereof | |
| CN107885822A (en) | The generation method and device of a kind of operation code | |
| CN104714923A (en) | Method and device for achieving equipment sharing | |
| KR20100062442A (en) | System and method for accessing to file storage | |
| Mandal et al. | A novel and flexible cloud architecture for data-centric applications | |
| CN1984149A (en) | Distributing system based on message and its telecommunication | |
| CA2638875C (en) | System and method for rendering presentation pages based on locality | |
| CN111913928A (en) | Distributed small file system, method, storage medium and electronic equipment for block storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C53 | Correction of patent of invention or patent application | ||
| CB02 | Change of applicant information |
Address after: 100085 Haidian District, Zhongguancun, South Street, No. 6,, building information, floor, No. 16 Applicant after: SI-TECH Information Technology Ltd. Address before: 100085, Beijing, Haidian District on the nine Street 9 digital science and Technology Plaza, two floor Applicant before: Beijing Digital China SI-TECH Information Technology Co., Ltd. |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: APPLICANT; FROM: BEIJING DIGITAL CHINA SI-TECH INFORMATION TECHNOLOGY LTD. TO: BEIJING SI-TECH INFORMATION TECHNOLOGY LTD. |
|
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20120125 |