WO2018094909A1 - 节点连接方法及分布式计算系统 - Google Patents

节点连接方法及分布式计算系统 Download PDF

Info

Publication number
WO2018094909A1
WO2018094909A1 PCT/CN2017/076025 CN2017076025W WO2018094909A1 WO 2018094909 A1 WO2018094909 A1 WO 2018094909A1 CN 2017076025 W CN2017076025 W CN 2017076025W WO 2018094909 A1 WO2018094909 A1 WO 2018094909A1
Authority
WO
WIPO (PCT)
Prior art keywords
node
mapping table
standby
module
active
Prior art date
Application number
PCT/CN2017/076025
Other languages
English (en)
French (fr)
Inventor
陈秋忠
刘华明
Original Assignee
网宿科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 网宿科技股份有限公司 filed Critical 网宿科技股份有限公司
Priority to EP17801324.9A priority Critical patent/EP3352433B1/en
Priority to US15/548,048 priority patent/US20180270145A1/en
Publication of WO2018094909A1 publication Critical patent/WO2018094909A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/02—Topology update or discovery
    • H04L45/021—Ensuring consistency of routing table updates, e.g. by using epoch numbers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/14—Session management
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/14—Session management
    • H04L67/141—Setup of application sessions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45595—Network integration; Enabling network access in virtual machine instances

Definitions

  • the embodiments of the present invention relate to the field of Internet technologies, and in particular, to a node connection method and a distributed computing system.
  • the in-memory database Redis (key-value database) supports the active and standby, and the official provides the redis-sentinel (the monitoring management, notification, and instance failover service of the Redis instance) for active and standby monitoring and active/standby switching, but because of the master device
  • the transfer does not allow the Redis client to always connect to the same ip address.
  • the known solutions are based on haproxy (reverse proxy software) + keepalived (cluster management to ensure high availability of cluster software) + sentinel combination, keepalived provides vip (virtual ip) for client connection and manage haproxy Failover (failover), haproxy determines whether redis is the master device and directly connects to the master device.
  • the sentinel performs active/standby monitoring and switching for redis.
  • the solution system is overly complex. Although a set of sentinel and haproxy can manage multiple redis masters and backups, a set of keepalived only one vip is used to map a set of Redis masters and backups. If you need to manage multiple redis masters and backups, you need multiple sets of keepalived. And if only one redis master and backup need to be managed, the system cost is too high.
  • the technical problem to be solved by the present invention is to overcome the defects of high cost and low efficiency of the active and standby management of the distributed computing system in the prior art, and to provide a system management cost reduction, a working efficiency of the client in the system, and an active/standby change of each server. Speed node connection method and distributed computing system.
  • a node connection method is provided for a distributed computing system, where the distributed computing system includes a plurality of clients and a server, wherein the server includes a mapping table, and the node connection method includes:
  • the client accesses the service side of the distributed computing system according to the node information
  • the client connects to the target node.
  • the application uses the mapping table of the server to store the mapping relationship.
  • the node information includes the node that the client needs to access, and the node may be a real node or a virtual node, and the client may be a node according to the node.
  • the information is searched for the corresponding target node in the mapping table, and the target node is a real node, that is, when the node recorded by the node information is a virtual node, the client needs to access the real node according to the real node corresponding to the virtual node in the mapping relationship.
  • the server may be a third end, independent of the server for computing storage.
  • RedisProxy (Redis Proxy), which uses virtual nodes to proxy real nodes.
  • a Redis cluster has multiple redis nodes, one of which has multiple masters. If a Redis cluster is regarded as a virtual Redis node. So RedisClient connected to the redis server does not need to consider whether the connected cluster is a single instance, so it is friendly to RedisClient.
  • the node connection method includes:
  • the system can monitor the changes of the master and backup. When there is a machine failure or a new machine is added, the system can quickly respond, improve the working efficiency of the client in the system and the change speed of the master and backup of each server.
  • the node includes a real node and a virtual node
  • the active/standby change includes a new active/standby switchover and an active/standby switchover in the distributed computing system
  • determining whether the active/standby change exists in the distributed computing system includes:
  • the master node and the standby node are real instances.
  • the active/standby switchover refers to the backup machine taking over the work of the host after the host fails, so that the system can continue to provide normal services.
  • the mapping relationship in the mapping table changes, the mapping table You need to replace the faulty primary node corresponding to the virtual node with the standby node. After the client listener mapping table changes, connect the updated standby node.
  • Adding the primary and backup means adding a new server to the original system.
  • the virtual node can be assigned to the real node in the newly added server, and the mapping relationship in the mapping table will also change.
  • the client listens. After the mapping table changes, connect to the updated standby node.
  • the monitoring of the active and standby can be implemented by the sentinel.
  • the client can determine whether the mapping table has changed.
  • the mapping relationship is changed, the latest correct real node is connected.
  • the acquiring the real node corresponding to the node information in the mapping table includes:
  • the node connection method further includes:
  • the real node that is accessed is changed by listening to the mapping table, and if so, the real node after the change is reconnected.
  • the node information includes information of a host name and a port number
  • the client identifies the node to be accessed according to a naming rule of a host name and a port name.
  • the node name can be a combination of a host name and a port, such as host001:1, host001:2, and host002:1. You can agree on the rules: host01 is virtual, and host001 is real (that is, the number is a two-digit virtual redis host name), the master node is host001:1, and the standby node is host002:1.
  • the distributed system includes a sentinel end for monitoring the active/standby change, and the mapping table is recorded in the zookeeper, and the node connection method includes:
  • the sentinel side determines whether an active/standby change has occurred, and if so, writes the changed mapping relationship to the zookeeper through a notification script.
  • the present invention further provides a distributed computing system, the distributed computing system includes a plurality of clients and a server, wherein the server includes a mapping table, and the server further includes a recording module, the client Including access module, acquisition module and connection module,
  • the recording module is configured to record node information and a mapping relationship between nodes in the mapping table
  • the access module is configured to access a service side of the distributed computing system according to the node information
  • the obtaining module is configured to acquire a target node corresponding to the node information in the mapping table
  • a connection module is used to connect to the target node.
  • the server further includes a determining module and an updating module.
  • the determining module is configured to determine whether there is a master/slave change in the distributed computing system, and if yes, invoke the update module;
  • the update module is configured to update a mapping relationship between the changed nodes to the mapping table
  • the acquiring module is configured to acquire the target node corresponding to the node information by using a latest mapping table.
  • the node includes a real node and a virtual node
  • the active/standby change includes a new active/standby switch and an active/standby switchover
  • the server further includes a processing module.
  • the determining module is further configured to determine whether the active/standby change is an active/standby switchover, and if yes, invoke the processing module;
  • the processing module is configured to switch a virtual node mapped node to a previous standby node in a mapping table
  • the determining module is further configured to determine whether the active/standby change is a new active/standby switch, and if yes, invoke the processing module;
  • the processing module is configured to add a node mapping relationship of the newly added active and standby nodes in the mapping table.
  • the master node and the standby node are real instances.
  • the client includes an identification module,
  • the identifying module is configured to: after the client accesses the mapping table, determine, according to the node information, whether the node to be accessed is a virtual node, and if yes, invoke the acquiring module to obtain a real node corresponding to the node to be accessed in the mapping table. And accessing, if otherwise, calling the connection module to directly access the to-be-accessed node.
  • the client includes a monitoring module
  • the monitoring module is configured to: after the client accesses the real node, determine whether the real node is changed by listening to the mapping table, and if yes, reconnect the changed real node.
  • the positive progress of the present invention is that the present invention can simplify the connection method of the redis client to the redis server in the distributed computing system, reduce the system deployment cost, and improve the performance of the redis client and the redis server connection.
  • the system and method of the present application do not need to use the existing conventional keepalived and haproxy complex systems, and only use the method of sentinel and zookeeper, and the sentinel performs the same active/standby switchover through the sentinel notification script to write the master information into the zookeeper. It can reduce system usage costs and increase efficiency.
  • FIG. 1 is a partial schematic structural diagram of a distributed computing system of Embodiment 1.
  • FIG. 2 is a flow chart showing a method of connecting nodes in the first embodiment.
  • FIG. 3 is a flow chart showing a method of connecting nodes in the third embodiment.
  • the embodiment provides a distributed computing system, where the distributed computing system includes a plurality of clients and a server.
  • the server includes a mapping table, and the mapping table is recorded in the zookeeper of the server.
  • the server further includes a recording module, and the client includes an access module, an obtaining module, an identifying module, a listening module, and a connecting module.
  • the recording module is configured to record node information and a mapping relationship between nodes in the mapping table.
  • the access module is configured to access a service side of the distributed computing system according to the node information, where the section The node where the point information is recorded may be a virtual node or a real node.
  • the identification module is configured to: after the client accesses the mapping table, determine, according to the node information, whether the node to be accessed is a virtual node, and if yes, invoke the acquiring module to obtain a real node corresponding to the node to be accessed in the mapping table, If the connection module is otherwise invoked, the access node is directly accessed.
  • the connection module is configured to access a real node.
  • the real node that is, the target node
  • the mapping table When the node in the node information is a real node, the connection module directly connects to the target node.
  • the node information includes information of a host name and a port number, and the client identifies the node to be accessed according to a naming rule of a host name and a port name.
  • the node name in the node information can be a combination of a host name and a port, such as host001:1, host001:2, and host002:1. You can agree on the rules: host01 is virtual, and host001 is real (that is, the number is a two-digit virtual redis host name), the master node is host001:1, and the standby node is host002:1.
  • the node name can be used to determine whether the node included in the node information is a real node or a virtual node.
  • the server has a sentinel function, that is, the server includes a sentinel end for monitoring the master/slave change, and the sentinel end includes a determining module and a processing module.
  • the determining module is configured to determine whether there is an active/standby switchover in the distributed computing system, and if yes, invoke the processing module.
  • the processing module is configured to switch the node mapped by the virtual node to the previous standby node in the mapping table, and after the host is collapsed, the virtual node needs to correspond to the node in the standby machine.
  • the master node and the standby node are real instances.
  • the sentinel side writes the correspondence to the zookeeper through the notification script.
  • the monitoring module is configured to: after the client accesses the real node, determine whether the real node has a change by monitoring the mapping table of the zookeeper, and if so, reconnect the real node after the change.
  • the embodiment further provides a node connection method, where the node connection method includes:
  • Step 100 Record node information and a mapping relationship between nodes in the mapping table.
  • the node information includes information of a host name and a port number, and the client identifies the node to be accessed according to a naming rule of a host name and a port name.
  • the node name in the node information can be a combination of a host name and a port, such as host001:1, host001:2, and host002:1. You can agree on the rules: host01 is virtual, and host001 is real (that is, the number is a two-digit virtual redis host name), the master node is host001:1, and the standby node is host002:1.
  • Step 101 The client accesses the service side of the distributed computing system according to the node information.
  • Step 102 Acquire the target node corresponding to the node information in the mapping table.
  • Step 103 The client connects to the target node.
  • the client can determine whether the accessed node is a real node or a virtual node according to the node information.
  • the client accesses the mapping table, and if it is a virtual node, obtains the real node corresponding to the node to be accessed and accesses the mapping table, and directly accesses the to-be-accessed node if it is a real node.
  • Step 104 The client listens to the service side Zookeeper to determine whether there is an active/standby change. If yes, execute step 105. Otherwise, execute step 104 again.
  • Step 105 The client disconnects the current target node, and returns to step 102.
  • the active/standby switchover includes active/standby switchover and new active/standby switchover.
  • the method switches the node mapped by the virtual node to the previous standby node in the mapping table, that is, after the host is collapsed, the virtual node needs to correspond to the node in the standby machine.
  • the method adds a new node mapping relationship between the active and standby nodes in the mapping table, and the client can connect the updated node according to the node information and the new mapping table.
  • the client obtains a change in the real node accessed by listening to the mapping table, and if so, reconnects the changed real node.
  • the node connection method and the distributed computing system of the embodiment can simplify the system structure of the distributed computing system, reduce the system management cost, improve the working efficiency of the client in the system, and change the speed of the active/standby of each server.
  • This embodiment is basically the same as Embodiment 1, except that:
  • the determining module is configured to determine whether there is a new active/standby in the distributed computing system, and if yes, invoke the processing module;
  • the processing module is configured to add a node mapping relationship of the newly added active and standby nodes in the mapping table.
  • the node connection method of this embodiment is different from that of Embodiment 1 in that:
  • the master node and the standby node are real instances.
  • the node connection method and the distributed computing system of the embodiment can simplify the system structure of the distributed computing system, reduce the system management cost, improve the working efficiency of the client in the system, and change the speed of the active/standby of each server.
  • this embodiment is basically the same as Embodiment 1, except that:
  • step 103 the method includes:
  • Step 200 The sentinel monitors whether there is a master/slave change in the distributed computing system. If yes, step 201 is performed, and if not, step 200 is performed again.
  • Step 201 Generate a notification script according to the change information.
  • Step 202 Execute the notification script, write the change information to the zookeeper, and perform step 200 again.
  • Embodiment 1 is further optimized on Embodiment 1.
  • the client disconnects the current connection and then connects to the latest node according to the mapping table.
  • the service side changes the information of the active/standby change. New, change information is written to zookeeper.
  • the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Hardware Redundancy (AREA)
  • Computer And Data Communications (AREA)
  • Multi Processors (AREA)
  • Debugging And Monitoring (AREA)

Abstract

本发明公开了一种节点连接方法及分布式计算系统，所述分布式计算系统包括若干客户端以及服务端，所述服务端包括映射表，所述节点连接方法包括：在所述映射表中记录节点信息以及节点间的映射关系；客户端根据节点信息访问分布式计算系统的服务侧；获取所述节点信息在所述映射表中对应的目标节点；所述客户端连接所述目标节点。本发明能够简化分布式计算系统的系统结构，降低系统管理成本，提高系统内客户端工作效率以及各服务器主备变更速度。

Description

节点连接方法及分布式计算系统 技术领域
本发明实施例涉及互联网技术领域，尤其涉及一种节点连接方法及分布式计算系统。
背景技术
内存数据库Redis(键值数据库)是支持主备的，并且官方提供了redis-sentinel(Redis实例的监控管理、通知和实例失效备援服务)进行主备监控及主备切换，但是因为主设备的转移无法使得Redis客户端始终连接同一个ip地址。目前，已知的方案有是采用haproxy(反向代理软件)+keepalived(集群管理中保证集群高可用的服务软件)+sentinel结合的方式，keepalived提供vip(虚拟ip)供客户端连接并且管理haproxy failover(故障切换)，haproxy判断redis是否为主设备并且直连主设备，sentinel对redis进行主备监控及切换。
现有的加速方案存在两个弊端：
1、该方案系统过度复杂。虽然一组sentinel跟haproxy可以对多个redis的主备进行管理，但是一组keepalived只有一个vip用来映射一组Redis主备， 如果需要管理多个redis主备，就需要多组keepalived。而且如果仅需要对一个redis主备进行管理的话，系统成本太高。
2、该方案必然导致性能问题，因为所有连接都经过haproxy转发，对于高速的Redis，相比HAProxy的长的等待时间来说确实会产生额外的负担。
发明内容
本发明要解决的技术问题是为了克服现有技术中分布式计算系统主备管理成本高且效率低下的缺陷，提供一种降低系统管理成本，提高系统内客户端工作效率以及各服务器主备变更速度的节点连接方法及分布式计算系统。
本发明是通过下述技术方案来解决上述技术问题：
一种节点连接方法，用于分布式计算系统，所述分布式计算系统包括若干客户端以及服务端，其特点在于，所述服务端包括映射表，所述节点连接方法包括：
在所述映射表中记录节点信息以及节点间的映射关系；
客户端根据节点信息访问分布式计算系统的服务侧；
获取所述节点信息在所述映射表中对应的目标节点；
所述客户端连接所述目标节点。
本申请利用服务端的映射表存储映射关系，节点信息中包括客户端需要访问的节点，所述节点可以为真实节点也可以为虚拟节点，客户端根据节点 信息在映射表中查找对应目标节点，所述目标节点为真实节点，也就是说，当节点信息记录的节点是虚拟节点时，客户端需要根据映射关系中虚拟节点对应的真实节点来访问真实节点。所述服务端可以为第三端，独立于计算存储用的服务器。
上述分布式计算系统实现了RedisProxy(Redis代理)的概念，利用虚拟节点代理真实节点，一个Redis集群有多个redis节点，其中一个master多个slave，如果把一个Redis集群当做是一个虚拟的Redis节点，这样RedisClient连接redis server就无需考虑所连接的是集群还是单实例，这样对RedisClient是友好的。
较佳地，所述节点连接方法包括：
判断所述分布式计算系统中是否存在主备变更，若是则将变更后节点间的映射关系更新至所述映射表中；
通过最新映射表获取所述节点信息对应的所述目标节点。
本申请中系统可以监听主备的变更，当存在机器故障或添加新机器时，系统能够快速的做出反应，提高系统内客户端工作效率以及各服务器主备变更速度。
较佳地，所述节点包括真实节点和虚拟节点，所述主备变更包括分布式计算系统新增主备和主备切换，所述判断所述分布式计算系统中是否存在主备变更包括：
判断主备变更是否为主备切换，若是则在映射表中将虚拟节点映射的主节点切换为之前的备节点；或，
判断主备变更是否为新增主备，若是则在映射表中添加新增主备的节点映射关系；
其中，主节点、备节点均为真实实例。
本申请中，主备切换是指当主机出故障宕机后，备用机接替主机的工作来使系统能够继续提供正常的服务，当出现这种情况是映射表中的映射关系发生改变，映射表需要将虚拟节点对应的出问题的主节点替换为备节点，客户端监听映射表发生变化后，连接更新的备节点。
新增主备是指在原有的系统中添加新的服务器，当出现这种情况时可以将虚拟节点分配给新增服务器中的真实节点，映射表中的映射关系也会发生改变，客户端监听映射表发生变化后，连接更新的备节点。
对主备的监听可以通过sentinel实现，当主备变更后映射表发生改变，客户端可以判断映射表是否发生改变，当判断映射关系变动后连接最新的正确的真实节点。
较佳地，所述获取所述节点信息在所述映射表中对应的真实节点包括：
客户端访问映射表；
客户端根据所述节点信息判断待访问节点是否为虚拟节点，若是则在所述映射表中获取待访问节点对应的真实节点并访问，若否则直接访问所述待 访问节点。
较佳地，所述节点连接方法还包括：
客户端访问真实节点后，通过监听所述映射表得到访问的真实节点是否有变更，若是则重连变更后的真实节点。
较佳地，所述节点信息包括主机名及端口号的信息，所述客户端根据主机名及端口名的命名规则识别待访问节点。
节点名可以为主机名跟端口的结合，例如host001:1，host001:2，host002:1。可以约定规则：host01为虚拟，host001为真实(即数字为两位数的为虚拟的redis主机名)，主节点为host001:1，备节点为host002:1。
较佳地，所述分布式系统包括用于监控主备变更的sentinel端，所述映射表记录于zookeeper，所述节点连接方法包括：
所述sentinel端判断是否发生主备变更，若是则通过通知脚本将变更后的映射关系写入zookeeper。
本发明还提供一种分布式计算系统，所述分布式计算系统包括若干客户端以及服务端，其特点在于，所述服务端包括映射表，所述服务端还包括记录模块，所述客户端包括访问模块、获取模块及连接模块，
所述记录模块用于在所述映射表中记录节点信息以及节点间的映射关系；
所述访问模块用于根据节点信息访问分布式计算系统的服务侧；
所述获取模块用于获取所述节点信息在所述映射表中对应的目标节点；
连接模块用于连接所述目标节点。
较佳地，所述服务端还包括判断模块及更新模块，
所述判断模块用于判断所述分布式计算系统中是否存在主备变更，若是则调用所述更新模块；
所述更新模块用于将变更后节点间的映射关系更新至所述映射表中；
所述获取模块用于通过最新映射表获取所述节点信息对应的所述目标节点。
较佳地，所述节点包括真实节点和虚拟节点，所述主备变更包括分布式计算系统新增主备和主备切换，所述服务端还包括处理模块，
所述判断模块还用于判断主备变更是否为主备切换，若是则调用所述处理模块；
所述处理模块用于在映射表中将虚拟节点映射的节点切换为之前的备节点；
或，
所述判断模块还用于判断主备变更是否为新增主备，若是则调用所述处理模块；
所述处理模块用于在映射表中添加新增主备的节点映射关系；
其中，主节点、备节点均为真实实例。
较佳地，所述客户端包括识别模块，
所述识别模块用于在客户端访问映射表后，根据所述节点信息判断待访问的节点是否为虚拟节点，若是则调用所述获取模块在所述映射表中获取待访问节点对应的真实节点并访问，若否则调用所述连接模块直接访问所述待访问节点。
较佳地，所述客户端包括监听模块，
所述监听模块用于在客户端访问真实节点后，通过监听所述映射表判断真实节点是否有变更，若是则重连变更后的真实节点。
本发明的积极进步效果在于：本发明能够简化分布式计算系统中redis客户端到redis服务端的连接方法，降低系统部署成本，提高redis客户端与redis服务端连接的性能。
具体地，本申请的系统及方法无需使用现有常规的keepalived和haproxy复杂系统，只采用sentinel和zookeeper的方式，sentinel进行同样主备切换通过sentinel的通知脚本将master信息写入zookeeper之中，不仅能够降低系统使用成本，还可以提高效率。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下 面描述中的附图为本发明的一些实施例，对于本领域普通技术人员而言，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为实施例1的分布式计算系统的局部结构示意图。
图2为实施例1的节点连接方法的流程图。
图3为实施例3的节点连接方法的流程图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例为本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
实施例1
本实施例提供一种分布式计算系统，所述分布式计算系统包括若干客户端以及服务端，所述服务端包括一个映射表，所述映射表记录于服务端的zookeeper。所述服务端还包括记录模块，所述客户端均包括访问模块、获取模块、识别模块、监听模块及连接模块。
所述记录模块用于在所述映射表中记录节点信息以及节点间的映射关系。
所述访问模块用于根据节点信息访问分布式计算系统的服务侧，所述节 点信息记录的节点可能为虚拟节点也可能为真实节点。
所述识别模块用于在客户端访问映射表后，根据所述节点信息判断待访问节点是否为虚拟节点，若是则调用所述获取模块在所述映射表中获取待访问节点对应的真实节点，若否则调用所述连接模块直接访问所述待访问节点。
所述连接模块用于访问真实节点，当节点信息中的节点为虚拟节点时，根据映射表获取真实节点，即目标节点。当节点信息中的节点为真实节点时，连接模块直接连接目标节点。
所述节点信息包括主机名及端口号的信息，所述客户端根据主机名及端口名的命名规则识别待访问节点。
节点信息中的节点名可以为主机名跟端口的结合，例如host001:1，host001:2，host002:1。可以约定规则：host01为虚拟，host001为真实(即数字为两位数的为虚拟的redis主机名)，主节点为host001:1，备节点为host002:1。
通过节点名可以判断出节点信息中包括的节点是真实节点还是虚拟节点。
参见图1，本实施例中，服务端具有sentinel功能，即服务端包括用于监控主备变更的sentinel端，所述sentinel端包括判断模块及处理模块。
所述判断模块用于判断所述分布式计算系统中是否存在主备切换，若是则调用所述处理模块。
所述处理模块用于在映射表中将虚拟节点映射的节点切换为之前的备节点，也就主机宕掉了后，虚拟节点需要与备机中节点对应上。
其中，主节点、备节点均为真实实例。
sentinel端通过通知脚本将对应关系写入zookeeper中。
所述监听模块用于在客户端访问真实节点后，通过监听zookeeper的映射表判断真实节点是否有变更，若是则重连变更后的真实节点。
利用上述分布式计算系统，本实施例还提供一种节点连接方法，所述节点连接方法包括：
步骤100、在所述映射表中记录节点信息以及节点间的映射关系。
所述节点信息包括主机名及端口号的信息，所述客户端根据主机名及端口名的命名规则识别待访问节点。
节点信息中的节点名可以为主机名跟端口的结合，例如host001:1，host001:2，host002:1。可以约定规则：host01为虚拟，host001为真实(即数字为两位数的为虚拟的redis主机名)，主节点为host001:1，备节点为host002:1。
步骤101、客户端根据节点信息访问分布式计算系统的服务侧。
步骤102、获取所述节点信息在所述映射表中对应的目标节点。
步骤103、所述客户端连接所述目标节点。
客户端根据节点信息能够判断访问的节点是真实节点还是虚拟节点，客 户端访问映射表，并若是虚拟节点则在所述映射表中获取待访问节点对应的真实节点并访问，若是真实节点则直接访问所述待访问节点。
步骤104、客户端监听服务侧Zookeeper判断是否存在主备变更，若是则执行步骤105，若否则再次执行步骤104。
步骤105、客户端断开当前目标节点的连接，返回步骤102。
所述主备变更包括主备切换和新增主备。当主备切换时，本方法在映射表中将虚拟节点映射的节点切换为之前的备节点，也就是主机宕掉了后，虚拟节点需要与备机中节点对应上。
当新增主备时，本方法在映射表中添加新增主备的节点映射关系，客户端可以按照节点信息及新的映射表连接更新后的节点。
所述客户端通过监听所述映射表得到访问的真实节点是否有变更，若是则重连变更后的真实节点。
本实施例的节点连接方法及分布式计算系统能够简化分布式计算系统的系统结构，降低系统管理成本，提高系统内客户端工作效率以及各服务器主备变更速度。
实施例2
本实施例与实施例1基本相同，不同之处仅在于：
所述判断模块用于判断所述分布式计算系统中是否存在新增主备，若是则调用所述处理模块；
所述处理模块用于在映射表中添加新增主备的节点映射关系。
利用上述分布式计算系统，本实施例的节点连接方法与实施例1的不同之处在于：
判断主备变更是否为新增主备，若是则在映射表中添加新增主备的节点映射关系；
其中，主节点、备节点均为真实实例。
本实施例的节点连接方法及分布式计算系统能够简化分布式计算系统的系统结构，降低系统管理成本，提高系统内客户端工作效率以及各服务器主备变更速度。
实施例3
参见图3，本实施例与实施例1基本相同，不同之处仅在于：
步骤103后包括：
步骤200、sentinel监听所述分布式计算系统中是否存在主备变更，若是则执行步骤201，若否则再次执行步骤200。
步骤201、根据变更信息生成通知脚本。
步骤202、执行所述通知脚本，将变更信息写入zookeeper中，再次执行步骤200。
本实施例在实施例1上进一步优化，当出现主备变更后客户端断开当前连接然后根据映射表连接最新的节点，另一方面服务侧将主备变更的信息更 新，变更信息写入zookeeper。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (12)

  1. 一种节点连接方法，用于分布式计算系统，所述分布式计算系统包括若干客户端以及服务端，其特征在于，所述服务端包括映射表，所述节点连接方法包括：
    在所述映射表中记录节点信息以及节点间的映射关系；
    客户端根据节点信息访问分布式计算系统的服务侧；
    获取所述节点信息在所述映射表中对应的目标节点；
    所述客户端连接所述目标节点。
  2. 如权利要求1所述的节点连接方法，其特征在于，所述节点连接方法包括：
    判断所述分布式计算系统中是否存在主备变更，若是则将变更后节点间的映射关系更新至所述映射表中；
    通过最新映射表获取所述节点信息对应的所述目标节点。
  3. 如权利要求2所述的节点连接方法，其特征在于，所述节点包括真实节点和虚拟节点，所述主备变更包括分布式计算系统新增主备和主备切换，所述判断所述分布式计算系统中是否存在主备变更包括：
    判断主备变更是否为主备切换，若是则在映射表中将虚拟节点映射的主节点切换为之前的备节点；或，
    判断主备变更是否为新增主备，若是则在映射表中添加新增主备的节点 映射关系；
    其中，主节点、备节点均为真实实例。
  4. 如权利要求1所述的节点连接方法，其特征在于，所述获取所述节点信息在所述映射表中对应的真实节点包括：
    客户端访问映射表；
    客户端根据所述节点信息判断待访问节点是否为虚拟节点，若是则在所述映射表中获取待访问节点对应的真实节点并访问，若否则直接访问所述待访问节点。
  5. 如权利要求4所述的节点连接方法，其特征在于，所述节点连接方法还包括：
    客户端访问真实节点后，通过监听所述映射表得到访问的真实节点是否有变更，若是则重连变更后的真实节点。
  6. 如权利要求1所述的节点连接方法，其特征在于，所述节点信息包括主机名及端口号的信息，所述客户端根据主机名及端口名的命名规则识别待访问节点。
  7. 如权利要求1所述的节点连接方法，其特征在于，所述分布式系统包括用于监控主备变更的sentinel端，所述映射表记录于zookeeper，所述节点连接方法包括：
    所述sentinel端判断是否发生主备变更，若是则通过通知脚本将变更后的 映射关系写入zookeeper。
  8. 一种分布式计算系统，所述分布式计算系统包括若干客户端以及服务端，其特征在于，所述服务端包括映射表，所述服务端还包括记录模块，所述客户端包括访问模块、获取模块及连接模块，
    所述记录模块用于在所述映射表中记录节点信息以及节点间的映射关系；
    所述访问模块用于根据节点信息访问分布式计算系统的服务侧；
    所述获取模块用于获取所述节点信息在所述映射表中对应的目标节点；
    连接模块用于连接所述目标节点。
  9. 如权利要求8所述的分布式计算系统，其特征在于，所述服务端还包括判断模块及更新模块，
    所述判断模块用于判断所述分布式计算系统中是否存在主备变更，若是则调用所述更新模块；
    所述更新模块用于将变更后节点间的映射关系更新至所述映射表中；
    所述获取模块用于通过最新映射表获取所述节点信息对应的所述目标节点。
  10. 如权利要求9所述的分布式计算系统，其特征在于，所述节点包括真实节点和虚拟节点，所述主备变更包括分布式计算系统新增主备和主备切换，所述服务端还包括处理模块，
    所述判断模块还用于判断主备变更是否为主备切换，若是则调用所述处理模块；
    所述处理模块用于在映射表中将虚拟节点映射的节点切换为之前的备节点；
    或，
    所述判断模块还用于判断主备变更是否为新增主备，若是则调用所述处理模块；
    所述处理模块用于在映射表中添加新增主备的节点映射关系；
    其中，主节点、备节点均为真实实例。
  11. 如权利要求8所述的分布式计算系统，其特征在于，所述客户端包括识别模块，
    所述识别模块用于在客户端访问映射表后，根据所述节点信息判断待访问节点是否为虚拟节点，若是则调用所述获取模块在所述映射表中获取待访问节点对应的真实节点并访问，若否则调用所述连接模块直接访问所述待访问节点。
  12. 如权利要求11所述的分布式计算系统，其特征在于，所述客户端包括监听模块，
    所述监听模块用于在客户端访问真实节点后，通过监听所述映射表判断真实节点是否有变更，若是则重连变更后的真实节点。
PCT/CN2017/076025 2016-11-28 2017-03-09 节点连接方法及分布式计算系统 WO2018094909A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP17801324.9A EP3352433B1 (en) 2016-11-28 2017-03-09 Node connection method and distributed computing system
US15/548,048 US20180270145A1 (en) 2016-11-28 2017-03-09 Node connection method and distributed computing system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611060923.1 2016-11-28
CN201611060923.1A CN106534328B (zh) 2016-11-28 2016-11-28 节点连接方法及分布式计算系统

Publications (1)

Publication Number Publication Date
WO2018094909A1 true WO2018094909A1 (zh) 2018-05-31

Family

ID=58357501

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/076025 WO2018094909A1 (zh) 2016-11-28 2017-03-09 节点连接方法及分布式计算系统

Country Status (4)

Country Link
US (1) US20180270145A1 (zh)
EP (1) EP3352433B1 (zh)
CN (1) CN106534328B (zh)
WO (1) WO2018094909A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112671601A (zh) * 2020-12-11 2021-04-16 航天信息股份有限公司 一种基于Zookeeper的接口监控系统及方法
CN112866035A (zh) * 2021-02-24 2021-05-28 紫光云技术有限公司 一种云平台上redis服务可指定从节点切换为主节点的方法
CN113992683A (zh) * 2021-10-25 2022-01-28 重庆紫光华山智安科技有限公司 实现同一集群中双网络有效隔离的方法、系统、设备及介质
CN114666202A (zh) * 2022-03-18 2022-06-24 中国建设银行股份有限公司 一种基于云数据库的主从切换的监控方法及装置

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106534328B (zh) * 2016-11-28 2020-01-31 网宿科技股份有限公司 节点连接方法及分布式计算系统
CN108833503B (zh) * 2018-05-29 2021-07-20 华南理工大学 一种基于ZooKeeper的Redis集群方法
US10922199B2 (en) * 2018-07-04 2021-02-16 Vmware, Inc. Role management of compute nodes in distributed clusters
CN110290163B (zh) * 2018-08-28 2022-03-25 新华三技术有限公司 一种数据处理方法及装置
CN111107120B (zh) * 2018-10-29 2022-09-02 亿阳信通股份有限公司 一种Redis集群的构建方法及系统
CN109800136A (zh) * 2018-12-06 2019-05-24 珠海西山居移动游戏科技有限公司 一种远程redis性能数据采样方法及其系统
CN109617761B (zh) * 2018-12-10 2020-02-21 北京明朝万达科技股份有限公司 一种主备服务器切换方法及装置
CN109639704B (zh) * 2018-12-26 2019-11-08 苏州沁游网络科技有限公司 一种主从式服务器系统应用方法、系统、服务器及存储介质
CN110324176A (zh) * 2019-05-29 2019-10-11 平安科技（深圳）有限公司 基于Redis的mqtt集群的监控方法、装置及存储介质
CN110224871B (zh) * 2019-06-21 2022-11-08 深圳前海微众银行股份有限公司 一种Redis集群的高可用方法及装置
CN110324253A (zh) * 2019-06-29 2019-10-11 江苏满运软件科技有限公司 流量控制方法、装置、存储介质及电子设备
CN111787055B (zh) * 2020-05-22 2021-07-27 中国科学院信息工程研究所 一种基于Redis且面向事务机制和多数据中心的数据分发方法和系统
CN114531688A (zh) * 2022-01-04 2022-05-24 宜兴市苏信智能技术发展研究中心 一种基于5g和区块链的无线组网方法
CN115426249B (zh) * 2022-11-02 2023-03-24 飞天诚信科技股份有限公司 一种Redis主从架构的高可用解决方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140149794A1 (en) * 2011-12-07 2014-05-29 Sachin Shetty System and method of implementing an object storage infrastructure for cloud-based services
CN103905530A (zh) * 2014-03-11 2014-07-02 浪潮集团山东通用软件有限公司 一种高性能全局负载均衡的分布式数据库数据路由方法
CN104636076A (zh) * 2013-11-15 2015-05-20 中国电信股份有限公司 一种用于云存储的分布式块设备驱动方法和系统
CN106534328A (zh) * 2016-11-28 2017-03-22 网宿科技股份有限公司 节点连接方法及分布式计算系统

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3494562B2 (ja) * 1997-10-15 2004-02-09 株式会社東芝 ネットワーク管理システム
JP4501096B2 (ja) * 1998-05-12 2010-07-14 オラクル・アメリカ・インコーポレーテッド 高可用性クラスタ仮想ディスク・システム
US7561531B2 (en) * 2005-04-19 2009-07-14 Intel Corporation Apparatus and method having a virtual bridge to route data frames
US7480780B2 (en) * 2005-04-19 2009-01-20 Hitachi, Ltd. Highly available external storage system
JP4738941B2 (ja) * 2005-08-25 2011-08-03 株式会社日立製作所 ストレージシステム及びストレージシステムの管理方法
JP4877921B2 (ja) * 2006-01-25 2012-02-15 株式会社日立製作所 ストレージシステム、記憶制御装置及び記憶制御装置のリカバリポイント検出方法
JP4757038B2 (ja) * 2006-01-25 2011-08-24 株式会社日立製作所 ストレージシステム及び記憶制御装置
US8266688B2 (en) * 2007-10-19 2012-09-11 Citrix Systems, Inc. Systems and methods for enhancing security by selectively opening a listening port when an incoming connection is expected
CN102467408B (zh) * 2010-11-12 2014-03-19 阿里巴巴集团控股有限公司 一种虚拟机数据的访问方法和设备
CN102143063B (zh) * 2010-12-29 2014-04-02 华为技术有限公司 集群系统中业务保护的方法和装置
US20130067095A1 (en) * 2011-09-09 2013-03-14 Microsoft Corporation Smb2 scaleout
US10372685B2 (en) * 2014-03-31 2019-08-06 Amazon Technologies, Inc. Scalable file storage service
US10242000B2 (en) * 2016-05-27 2019-03-26 International Business Machines Corporation Consistent utility-preserving masking of a dataset in a distributed environment

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140149794A1 (en) * 2011-12-07 2014-05-29 Sachin Shetty System and method of implementing an object storage infrastructure for cloud-based services
CN104636076A (zh) * 2013-11-15 2015-05-20 中国电信股份有限公司 一种用于云存储的分布式块设备驱动方法和系统
CN103905530A (zh) * 2014-03-11 2014-07-02 浪潮集团山东通用软件有限公司 一种高性能全局负载均衡的分布式数据库数据路由方法
CN106534328A (zh) * 2016-11-28 2017-03-22 网宿科技股份有限公司 节点连接方法及分布式计算系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZENG, QUANJUN ET AL.: "A Design Scheme of Redis Cluster Management", SCIENCEPAPER ONLINE, 1 March 2014 (2014-03-01), pages 2 - 9, XP009506475 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112671601A (zh) * 2020-12-11 2021-04-16 航天信息股份有限公司 一种基于Zookeeper的接口监控系统及方法
CN112671601B (zh) * 2020-12-11 2023-10-31 航天信息股份有限公司 一种基于Zookeeper的接口监控系统及方法
CN112866035A (zh) * 2021-02-24 2021-05-28 紫光云技术有限公司 一种云平台上redis服务可指定从节点切换为主节点的方法
CN113992683A (zh) * 2021-10-25 2022-01-28 重庆紫光华山智安科技有限公司 实现同一集群中双网络有效隔离的方法、系统、设备及介质
CN113992683B (zh) * 2021-10-25 2024-02-13 重庆紫光华山智安科技有限公司 实现同一集群中双网络有效隔离的方法、系统、设备及介质
CN114666202A (zh) * 2022-03-18 2022-06-24 中国建设银行股份有限公司 一种基于云数据库的主从切换的监控方法及装置
CN114666202B (zh) * 2022-03-18 2024-04-26 中国建设银行股份有限公司 一种基于云数据库的主从切换的监控方法及装置

Also Published As

Publication number Publication date
EP3352433A1 (en) 2018-07-25
CN106534328B (zh) 2020-01-31
EP3352433B1 (en) 2019-10-02
CN106534328A (zh) 2017-03-22
EP3352433A4 (en) 2018-10-31
US20180270145A1 (en) 2018-09-20

Similar Documents

Publication Publication Date Title
WO2018094909A1 (zh) 节点连接方法及分布式计算系统
US11868323B2 (en) Orchestrated disaster recovery
US20200257593A1 (en) Storage cluster configuration change method, storage cluster, and computer system
US8954391B2 (en) System and method for supporting transient partition consistency in a distributed data grid
US9817721B1 (en) High availability management techniques for cluster resources
US9507678B2 (en) Non-disruptive controller replacement in a cross-cluster redundancy configuration
US9092460B2 (en) System and method for using a gridlink data source to connect an application server with a clustered database
JP7270755B2 (ja) 分散システムでのメタデータルーティング
US20130007253A1 (en) Method, system and corresponding device for load balancing
US11206173B2 (en) High availability on a distributed networking platform
GB2407887A (en) Automatically modifying fail-over configuration of back-up devices
JP2004537126A (ja) 高可用性クラスターの仮想サーバ・システム
JP2007503628A (ja) クラスタ化されたコンピューティングシステムにおける高速なアプリケーション通知
US20200019313A1 (en) Hard disk management method and system
WO2010123510A1 (en) Active-active support of virtual storage management in a storage area network ("san")
CN112953982A (zh) 一种服务处理的方法、服务配置的方法以及相关装置
CN112492022A (zh) 提高数据库可用性的集群、方法、系统及存储介质
CN113849136B (zh) 一种基于国产平台的自动化fc块存储处理方法和系统
CN114003350A (zh) 超融合系统的数据分配方法和系统
CN117632374A (zh) 容器镜像的读取方法、介质、装置和计算设备
US10305987B2 (en) Method to syncrhonize VSAN node status in VSAN cluster
CN116112569B (zh) 微服务调度方法及管理系统
CN115470303B (zh) 一种数据库访问方法、装置、系统、设备及可读存储介质
US9632890B2 (en) Facilitating N-way high availability storage services
WO2023029485A1 (zh) 数据处理方法、装置、计算机设备及计算机可读存储介质

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 15548048

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17801324

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE