CN104519130B - A kind of data sharing caching method across IDC - Google Patents

A kind of data sharing caching method across IDC Download PDF

Info

Publication number
CN104519130B
CN104519130B CN201410784343.1A CN201410784343A CN104519130B CN 104519130 B CN104519130 B CN 104519130B CN 201410784343 A CN201410784343 A CN 201410784343A CN 104519130 B CN104519130 B CN 104519130B
Authority
CN
China
Prior art keywords
data
write
local
internet
centers
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201410784343.1A
Other languages
Chinese (zh)
Other versions
CN104519130A (en
Inventor
张博
余效伟
李旭阳
胡道生
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BEIJING SINOIOV VEHICLE NETWORK TECHNOLOGY Co Ltd
Original Assignee
BEIJING SINOIOV VEHICLE NETWORK TECHNOLOGY Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BEIJING SINOIOV VEHICLE NETWORK TECHNOLOGY Co Ltd filed Critical BEIJING SINOIOV VEHICLE NETWORK TECHNOLOGY Co Ltd
Priority to CN201410784343.1A priority Critical patent/CN104519130B/en
Publication of CN104519130A publication Critical patent/CN104519130A/en
Application granted granted Critical
Publication of CN104519130B publication Critical patent/CN104519130B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

The present invention discloses a kind of data sharing caching method across IDC, and the method comprising the steps of：(1) local shared buffer memory, initialization data are established；(2) register, establish the shared buffer memory network connection of each Internet data center；(3) data write-in and synchronization are carried out；(4) data, services are provided to other Internet data centers.The shared buffer memory of multiple Internet data centers is connected as an entirety and carries out data write-in and synchronization by the present invention, and provide data, services to other Internet data centers, so the data of multiple Internet data centers can be made to complete data syn-chronization in a short time, ensure the uniformity of data, ensure that various regions Internet data center can obtain the data in the whole nation and provide data, services with this, significantly improve service quality and the service speed of response.

Description

A kind of data sharing caching method across IDC
Technical field
The present invention relates to the technical field of internet data processing, is cached more particularly to a kind of data sharing across IDC Method, i.e., the shared slow of data is carried out between different Internet data centers (IDC, Internet Data Center) Deposit.
Background technology
The shared buffer memory of internet services provides data support for real time service, is typically based on memory database, such as Redis, mongodb etc. are developed, and can be inquired about with quick response, are used extensively for Internet enterprises.Cached across IDC data sharing Method services, needed the field of comprehensive data on a large scale for requiring to provide, especially such as field of traffic, for it is particularly significant.
The mentality of designing of the existing data sharing caching method across IDC is as follows：
1. based on local data, regularly synchronous strange land data, such as strange land data of 24 hours Integral synchronous.
2. service is only for local service, it is impossible to the data of other Internet data centers of real-time synchronization.
Therefore, across IDC data sharing caching method there is obvious technical deficiency in the above：
1. shared buffer memory only supports local service；
2. it is unable to real-time synchronization strange land data center data；
3. associated service can not be provided before no synchronous strange land data；
4. data consistency is poor between different pieces of information center；
5. being limited by wide area network transmission, efficiency of transmission is not high；
6. lack real-time, or only synchronous static data, such as user basic information, it is impossible to synchronous user's real time position Multidate information.
But with the development and progress of human society, people are more and more using automobile as the vehicles, road On automobile increase swift and violent, the data between the Internet data center of different zones usually need to be transmitted in real time, for The user of different zones checks analysis.
The content of the invention
The technology of the present invention solves problem：Overcome the deficiencies in the prior art, there is provided a kind of data sharing across IDC caches Method, it can make the data of multiple Internet data centers complete data syn-chronization in a short time, ensure the uniformity of data, Ensure various regions Internet data center can obtain the whole nation data and provide data, services with this, significantly improve service quality and Service response speed.
The present invention technical solution be：This data sharing caching method across IDC, this method include following step Suddenly：
(1) local shared buffer memory, initialization data are established；
(2) register, establish the shared buffer memory network connection of each Internet data center；
(3) data write-in and synchronization are carried out；
(4) data, services are provided to other Internet data centers.
The shared buffer memory of multiple Internet data centers is connected as an entirety and carries out data write-in and synchronization by the present invention, And data, services are provided to other Internet data centers, so the data of multiple Internet data centers can be made in the short time Interior completion data syn-chronization, ensures the uniformity of data, ensure various regions Internet data center can obtain the data in the whole nation and with This provides data, services, significantly improves service quality and the service speed of response.
A kind of data sharing caching method across IDC is additionally provided, this method comprises the following steps：
(I) start；
(II) configuration information is obtained；
(III) system is initialized, establishes routing table；
(IV) network connection is initialized；
(V) local data base is connected；
(VI) service is started；
(VII) judge whether respond request terminates, be then to perform step (XIV), otherwise perform step (VIII)；
(VIII) request is monitored, if read requests then perform step (IX), if write request then performs step (XI)；
(IX) local data base is inquired about；
(X) respond request, jump procedure (VII)；
(XI) determine whether local request, be then to perform step (XII), otherwise perform step (XIII)；
(XII) route shunting, local data write other Internet data centers；
(XIII) local data base, jump procedure (X) are write；
(XIV) terminate.
Brief description of the drawings
Fig. 1 is the flow chart according to a preferred embodiment of the data sharing caching method across IDC of the present invention.
Embodiment
This data sharing caching method across IDC, this method comprise the following steps：
(1) local shared buffer memory, initialization data are established；
(2) register, establish the shared buffer memory network connection of each Internet data center；
(3) data write-in and synchronization are carried out；
(4) data, services are provided to other Internet data centers.
The shared buffer memory of multiple Internet data centers is connected as an entirety and carries out data write-in and synchronization by the present invention, And data, services are provided to other Internet data centers, so the data of multiple Internet data centers can be made in the short time Data syn-chronization is completed in (being 50ms during normal network conditions), ensures the uniformity of data, ensures various regions Internet data center The data in the whole nation can be obtained and provide data, services with this, significantly improve service quality and the service speed of response.
In addition, initialization data is from other Internet data center's synchrodatas in the step (1).Certainly, also may be used To take other manner to carry out initialization data.
In addition, the step (3) include it is following step by step：
(3.1) logical relation between each Internet data center is determined, determines whether local number according to logical relation According to needing to be synchronized to those data centers, routing table is established；
(3.2) data access, by routing table, judge to need the data center write；
(3.3) table of query and routing, routing channel is selected；
(3.4) local shared buffer memory is write；
(3.5) by route technology reading cache data, data are distributed；
(3.6) pass through data platform development kit DPDK servers (i.e. Distribution Center), change original header, be sent to it The DPDK servers of its Internet data center；
(3.7) the DPDK servers of other Internet data centers receive message, revert to original header, write local common Enjoy caching.
In addition, the step (3) also include it is following step by step：
(3.8) inquiry service is provided；
(3.9) real-time synchronization, when local data center has write operation, by the operation name and data of the operation, sequence Row turn to byte streams, are transmitted；Unserializing obtains operation data after strange land data center receives the message, performs the operation Complete synchronous.
As shown in figure 1, additionally providing a kind of data sharing caching method across IDC, this method comprises the following steps：
(I) start；(this step not shown in Fig. 1)
(II) configuration information is obtained；
(III) system is initialized, establishes routing table；
(IV) network connection is initialized；
(V) local data base is connected；
(VI) service is started；
(VII) judge whether respond request terminates, be then to perform step (XIV), otherwise perform step (VIII)；
(VIII) request is monitored, if read requests then perform step (IX), if write request then performs step (XI)；(now the data write-in of other Internet data centers is local)
(IX) local data base is inquired about；
(X) respond request, jump procedure (VII)；
(XI) determine whether local request, be then to perform step (XII), otherwise perform step (XIII)；
(XII) route shunting, local data write other Internet data centers；
(XIII) local data base, jump procedure (X) are write；
(XIV) terminate.
In addition, in the step (XII)：Data distribution center is distributed to other data centers according to routing rule；It is other Data center listens to new order, parses the order, and performs the order in place data center, completes data syn-chronization.
Relative to prior art, this method has following innovative point：
1. it is based on memory database；
2. it is based upon the routing mechanism that shared buffer memory establishes data distribution；
3. ensureing the high-speed of shared buffer memory data syn-chronization based on DPDK technologies, the distribution of packet is focused on into specialty On the webserver, coordinate compression algorithm, be remarkably improved the utilization rate of server and bandwidth；
4. plug-in unitization designs, it can be selected with Data replication and distribution strategy by user, only influence efficiency, do not influence function Realize.
It is described above, be only presently preferred embodiments of the present invention, any formal limitation not made to the present invention, it is every according to Any simple modification, equivalent change and modification made according to the technical spirit of the present invention to above example, still belong to the present invention The protection domain of technical scheme.

Claims (4)

  1. A kind of 1. data sharing caching method across IDC, it is characterised in that：This method comprises the following steps：
    (1) local shared buffer memory, initialization data are established；
    (2) register, establish the shared buffer memory network connection of each Internet data center；
    (3) data write-in and synchronization are carried out；
    (4) data, services are provided to other Internet data centers；
    Initialization data is from other Internet data center's synchrodatas in the step (1)；
    The step (3) include it is following step by step：
    (3.1) logical relation between each Internet data center is determined, determines whether that local data needs according to logical relation Those data centers are synchronized to, establish routing table；
    (3.2) data access, by routing table, judge to need the data center write；
    (3.3) table of query and routing, routing channel is selected；
    (3.4) local shared buffer memory is write；
    (3.5) by route technology reading cache data, data are distributed；
    (3.6) pass through data platform development kit DPDK servers, change original header, be sent to other Internet data centers DPDK servers；
    (3.7) the DPDK servers of other Internet data centers receive message, revert to original header, write local shared slow Deposit.
  2. 2. the data sharing caching method according to claim 1 across IDC, it is characterised in that：The step (3) also includes Below step by step：
    (3.8) inquiry service is provided；
    (3.9) real-time synchronization, when local data center has write operation, by the operation name and data of the operation, serializing Flow, be transmitted for byte；Unserializing obtains operation data after strange land data center receives the message, performs operation completion It is synchronous.
  3. A kind of 3. data sharing caching method across IDC, it is characterised in that：This method comprises the following steps：
    (I) start；
    (II) configuration information is obtained；
    (III) system is initialized, establishes routing table；
    (IV) network connection is initialized；
    (V) local data base is connected；
    (VI) service is started；
    (VII) judge whether respond request terminates, be then to perform step (XIV), otherwise perform step (VIII)；
    (VIII) request is monitored, if read requests then perform step (IX), if write request then performs step (XI)；
    (IX) local data base is inquired about；
    (X) respond request, jump procedure (VII)；
    (XI) determine whether local request, be then to perform step (XII), otherwise perform step (XIII)；
    (XII) route shunting, local data write other Internet data centers；
    (XIII) local data base, jump procedure (X) are write；
    (XIV) terminate.
  4. 4. the data sharing caching method according to claim 3 across IDC, it is characterised in that：In the step (XII)： Data distribution center is distributed to other data centers according to routing rule；Other data centers listen to new order, and parsing should Order, and the order is performed in place data center, complete data syn-chronization.
CN201410784343.1A 2014-12-16 2014-12-16 A kind of data sharing caching method across IDC Active CN104519130B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410784343.1A CN104519130B (en) 2014-12-16 2014-12-16 A kind of data sharing caching method across IDC

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410784343.1A CN104519130B (en) 2014-12-16 2014-12-16 A kind of data sharing caching method across IDC

Publications (2)

Publication Number Publication Date
CN104519130A CN104519130A (en) 2015-04-15
CN104519130B true CN104519130B (en) 2018-02-27

Family

ID=52793833

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410784343.1A Active CN104519130B (en) 2014-12-16 2014-12-16 A kind of data sharing caching method across IDC

Country Status (1)

Country Link
CN (1) CN104519130B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106933547B (en) 2015-12-29 2020-12-01 阿里巴巴集团控股有限公司 Global information acquisition and processing method, device and updating system
CN106933550B (en) 2015-12-29 2021-01-08 阿里巴巴集团控股有限公司 Global information obtaining, processing and updating method, device and system
CN106933548B (en) 2015-12-29 2021-01-12 阿里巴巴集团控股有限公司 Global information obtaining, processing and updating method, device and system
CN106101280B (en) * 2016-08-18 2019-01-22 无锡华云数据技术服务有限公司 A kind of network information synchronization update method between data center
CN109213766B (en) * 2018-09-12 2021-03-26 福建天泉教育科技有限公司 Data zero-delay processing method and terminal
CN109522353A (en) * 2018-11-09 2019-03-26 福建南威软件有限公司 The trans-regional shared implementation method of electronics license based on routing table
CN109788060B (en) * 2019-01-22 2021-05-07 曲阜师范大学 Cloud data sharing and updating method based on block chain
CN110809051B (en) * 2019-11-11 2020-11-13 广州华多网络科技有限公司 Service data processing method and system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103078957A (en) * 2013-02-01 2013-05-01 北京航空航天大学 Data center mirror image distribution system supporting IDC (Internet Data Center) domain functions
CN103685486A (en) * 2013-12-02 2014-03-26 中国科学院计算技术研究所 Distributed system monitoring method stepping over data center clusters and system
CN103763368A (en) * 2014-01-20 2014-04-30 浪潮电子信息产业股份有限公司 Cross-data-center data synchronism method
CN103945004A (en) * 2014-05-06 2014-07-23 中国联合网络通信集团有限公司 Dispatching method and system for data between data centers

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103078957A (en) * 2013-02-01 2013-05-01 北京航空航天大学 Data center mirror image distribution system supporting IDC (Internet Data Center) domain functions
CN103685486A (en) * 2013-12-02 2014-03-26 中国科学院计算技术研究所 Distributed system monitoring method stepping over data center clusters and system
CN103763368A (en) * 2014-01-20 2014-04-30 浪潮电子信息产业股份有限公司 Cross-data-center data synchronism method
CN103945004A (en) * 2014-05-06 2014-07-23 中国联合网络通信集团有限公司 Dispatching method and system for data between data centers

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"跨IDC ycache原理和配置说明";码迷;《码迷》;20141103;全文 *

Also Published As

Publication number Publication date
CN104519130A (en) 2015-04-15

Similar Documents

Publication Publication Date Title
CN104519130B (en) A kind of data sharing caching method across IDC
US9264511B2 (en) Content sharing among mobile terminals
CN103493455B (en) Use the global traffic management of modified host name
CN103812849B (en) A kind of local cache update method, system, client and server
CN103457993B (en) Local cache device and the method that content caching service is provided
CN103036977B (en) Business pushing method and pushing system based on content distribution network
CN105868231A (en) Cache data updating method and device
CN103581248A (en) Method and device for providing content distribution network CDN service
CN103825919B (en) A kind of method, the apparatus and system of data cached resource
CN103139252B (en) The implementation method that a kind of network proxy cache is accelerated and device thereof
CN103326925B (en) A kind of information push method and device
AU2016298291A1 (en) Methodology of a coordinate mapping system
CN106357539A (en) Data acquisition method and equipment
CN108901035A (en) The recognition methods of internet-of-things terminal and device
CN102572012A (en) Message processing method, switchboard and system
CN105681199A (en) Method and device for processing message data in vehicular bus
CN103001967B (en) Source station server and service push method
CN101895550B (en) Cache accelerating method for compatibility of dynamic and static contents of internet website
CN107566270A (en) The processing method and processing device that a kind of resource accesses
CN107070978A (en) A kind of inter-network acts on behalf of the method, system and proxy server of web map service
CN106569929A (en) Real-time data access method and system for monitoring system
CN105653717A (en) Information sharing method and device
CN106789655A (en) The sending method and device of a kind of advertising of route message
CN202103713U (en) Visiting system
CN106371122B (en) RTK base station search method based on cloud locating platform

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant