WO2019095577A1 - Method and device for processing data request, computer apparatus, and storage medium - Google Patents

Method and device for processing data request, computer apparatus, and storage medium Download PDF

Info

Publication number
WO2019095577A1
WO2019095577A1 PCT/CN2018/077132 CN2018077132W WO2019095577A1 WO 2019095577 A1 WO2019095577 A1 WO 2019095577A1 CN 2018077132 W CN2018077132 W CN 2018077132W WO 2019095577 A1 WO2019095577 A1 WO 2019095577A1
Authority
WO
WIPO (PCT)
Prior art keywords
cluster
identifier
target
data
response
Prior art date
Application number
PCT/CN2018/077132
Other languages
French (fr)
Chinese (zh)
Inventor
兰东平
Original Assignee
平安科技(深圳)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技(深圳)有限公司 filed Critical 平安科技(深圳)有限公司
Publication of WO2019095577A1 publication Critical patent/WO2019095577A1/en

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06Management of faults, events, alarms or notifications
    • H04L41/0654Management of faults, events, alarms or notifications using network fault recovery
    • H04L41/0663Performing the actions predefined by failover planning, e.g. switching to standby network elements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0805Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability
    • H04L43/0817Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/10Active monitoring, e.g. heartbeat, ping or trace-route
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/104Peer-to-peer [P2P] networks
    • H04L67/1044Group management mechanisms 
    • H04L67/1046Joining mechanisms
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/104Peer-to-peer [P2P] networks
    • H04L67/1044Group management mechanisms 
    • H04L67/1048Departure or maintenance mechanisms

Definitions

  • the present application relates to a data request processing method, apparatus, computer device, and storage medium.
  • a cluster is a whole of multiple servers connected by a network, usually a high-speed LAN. It provides services to external clients in a unified manner, and it looks like a server.
  • the inventor realized that when the cluster capacity is insufficient and a new cluster is needed, since the domain name of the new cluster is different from the domain name of the original cluster, the data of the original cluster needs to be migrated to the new cluster, the workload is large, and the main cluster is When a fault occurs and the standby cluster needs to be enabled for service, the domain name resolution scheme needs to be manually changed, and the timeliness is poor.
  • a data request processing method, apparatus, computer device, and storage medium are provided.
  • a data request processing method includes:
  • a data request processing apparatus includes:
  • a data request receiving module configured to receive a data request sent by the client, where the data request carries a target data identifier
  • the cluster obtaining module is configured to obtain a corresponding primary cluster and a standby cluster according to the target data identifier, and the operation information obtaining module is configured to obtain the running information of the primary cluster, where the running information includes normal operation or abnormal operation;
  • a first response request sending module if the primary cluster runs abnormally, sending a first response request to the standby cluster according to the data request, where the first response request carries the target data identifier;
  • the response information receiving module is configured to receive first data response information sent by the standby cluster according to the first response request, and return the first data response information to the client.
  • a computer apparatus comprising a memory and one or more processors having stored therein computer readable instructions that, when executed by a processor, implement the steps of a data request processing method provided in any one of the embodiments of the present application. .
  • One or more non-transitory computer readable storage mediums storing computer readable instructions, when executed by one or more processors, cause one or more processors to be implemented in any one embodiment of the present application The steps of the data request processing method provided.
  • FIG. 1 is an application environment diagram of a data request processing method provided in accordance with one or more embodiments.
  • FIG. 2 is a flow diagram of a data request processing method in accordance with one or more embodiments.
  • FIG. 3 is a flowchart of obtaining a corresponding primary cluster and a standby cluster according to target data identification according to one or more embodiments.
  • FIG. 4 is a flow diagram of obtaining a corresponding primary cluster and standby cluster based on target key values in accordance with one or more embodiments.
  • FIG. 5 is a flow chart of a data request processing method in another embodiment.
  • FIG. 6 is a flow diagram of obtaining operational information for a primary cluster in accordance with one or more embodiments.
  • FIG. 7 is a block diagram of a data request processing apparatus in accordance with one or more embodiments.
  • FIG. 8 is a block diagram of a cluster get module in accordance with one or more embodiments.
  • FIG. 9 is a block diagram of a clustering unit in accordance with one or more embodiments.
  • Figure 10 is a block diagram of a data request processing apparatus in another embodiment.
  • 11 is a block diagram of a run information acquisition module in accordance with one or more embodiments.
  • Figure 12 is a block diagram of a computer device in accordance with one or more embodiments.
  • first may be referred to as a second primary cluster without departing from the scope of the present application, and similarly, the second primary cluster may be referred to as a first primary cluster.
  • FIG. 1 is an application environment diagram of a data request processing method provided in an embodiment.
  • the client 10, the computer device 20, and the first main cluster 31 are included.
  • the first standby cluster 32 corresponding to the first primary cluster 31, the second primary cluster 41, and the second standby cluster 42 corresponding to the second primary cluster 41. It can be connected via the network.
  • the primary cluster and the standby cluster are generally distributed in different cities, so that when the primary cluster fails, the standby cluster responds to the service.
  • the domain names of the first primary cluster 31 and the second primary cluster are different, but in the external service, an identical domain name may be set, and the domain name resolution result of the same domain name points to the IP address of the computer device 20.
  • the computer device 20 When the client issues a data request, for example, requesting to download a file, the computer device 20 receives the data request, obtains the corresponding primary cluster domain name according to the target data identifier in the data request, and then sends a response to the primary cluster according to the primary cluster domain name and the data request.
  • the request causes the corresponding primary cluster to respond and return a response message, and sends the response information to the client.
  • the proxy server sends a response request to the standby cluster according to the standby cluster domain name and the data request, and receives the response information returned by the standby cluster and sends the response information to the client.
  • the computer device 20 may be a stand-alone physical server or terminal, or may be a server cluster composed of multiple physical servers, and may be a cloud server that provides basic cloud computing services such as a cloud server, a cloud database, a cloud storage, and a CDN.
  • the client can be a smartphone, a tablet, a laptop, a desktop computer, a smart speaker, a smart watch, a server, etc., but is not limited thereto.
  • the primary cluster 41 and the second standby cluster 42 can be connected through a network. There may be one or more clusters, for example, there may be 10, which is not limited by the embodiment of the present application.
  • a data request processing method is provided, which may be applied to the computer device 20, and may specifically include the following steps:
  • Step S202 receiving a data request sent by the client, where the data request carries the target data identifier.
  • the data request may include requesting to download data in the cluster, requesting deletion of data in the cluster, or requesting one or more of uploading data to the cluster.
  • the data request also carries the data to be uploaded.
  • the target data identifier can be represented by at least one of data and text.
  • One or more clusters are served externally through a uniform access address.
  • the client receives the access address, such as a domain name or an IP address of a computer device, and sends a data request to the computer device based on the access address.
  • the data request is parsed by a DNS (Domain Name System) to obtain an IP address of the computing device 20 to route the data request to the computer device 20 when the input access address is an IP address.
  • the client sends a data request to the computer device 20 based on the IP address.
  • DNS Domain Name System
  • Step S204 obtaining a corresponding primary cluster and a standby cluster according to the target data identifier.
  • the correspondence between the data identifier and the cluster may be set in advance, and the correspondence may be a direct correspondence.
  • the data identifier a corresponds to the b cluster. It can also be an indirect correspondence.
  • the calculation rule and the correspondence between the value and the cluster may be set. After the data identifier is obtained, the data identifier is calculated according to the preset calculation rule, and the corresponding value is obtained, and the corresponding primary cluster is obtained according to the correspondence between the value and the cluster. Standby cluster.
  • step S206 the running information of the primary cluster is obtained, and the running information includes a normal operation or an abnormal operation.
  • the running information of the primary cluster can be obtained actively or automatically by the primary cluster.
  • the primary cluster can periodically send heartbeat information to the computer device. If the computer does not receive the heartbeat information of the cluster within a certain period of time, it indicates that the primary cluster is abnormal.
  • the data request may be sent to the primary cluster, and if the response of the primary cluster is not received, The primary cluster is running abnormally.
  • Step S208 If the primary cluster runs abnormally, send a first response request to the standby cluster according to the data request, where the first response request carries the target data identifier.
  • the primary cluster abnormality means that the primary cluster cannot respond. If the primary cluster runs abnormally, the first response request is sent to the standby cluster according to the data request. For example, when the data request is downloading data, the first response request for requesting the download data identifier to be the target data identifier is sent to the standby cluster, so that the standby cluster obtains the target data according to the target data identifier, and returns the target data to the computer device. When the data request is to delete data, the first response request for deleting the data identifier to the target data identifier is sent to the standby cluster, so that the standby cluster finds the target data according to the target data identifier, and deletes the target data. When the data request is to store data, the target data identifier and the target data are sent to the standby cluster, so that the standby cluster corresponds to the storage target data and the target data identifier.
  • the second response request is sent to the primary cluster according to the data request, the primary cluster responds and returns the second response information, and the computer device returns the second response information to the client.
  • the primary cluster and the standby cluster can be synchronized by the remote synchronization program, and the remote synchronization program reads the primary cluster operation record, and synchronizes the operation of the primary cluster to the standby cluster according to the primary cluster operation record.
  • Step S210 Receive first data response information sent by the standby cluster according to the first response request, and return the first data response information to the client.
  • the first data response information is information returned by the standby cluster according to the first response request.
  • the standby cluster obtains the target data according to the target data identifier, and returns the target data to the computer device.
  • the standby cluster finds the target data according to the target data identifier, and deletes the target data and returns the first data response information that is successfully deleted.
  • the standby cluster returns the storage first data response information corresponding to the storage target data and the target data identifier.
  • the standby cluster when the first response request is to download data, the standby cluster returns data to be downloaded.
  • the standby cluster is configured. Returns the first response message requesting an error.
  • the data request processing method When receiving the data request sent by the client, the data request processing method obtains the corresponding primary cluster and the standby cluster according to the target data identifier, and sends a first response request to the standby cluster when the primary cluster runs abnormally, and receives the standby cluster. And transmitting the first data response information according to the first response request to the client.
  • the corresponding primary cluster and the standby cluster are obtained according to the target data identifier, and when the primary cluster runs abnormally, the corresponding standby cluster responds according to the data request, so that the service can be externally accessed through the unified access address.
  • the cluster does not need to migrate data, which reduces the workload of the migration and can respond to the standby cluster in time when the primary cluster fails.
  • step S204 is to obtain a corresponding primary cluster and a standby cluster according to the target data identifier, including:
  • Step S302 performing a hash operation according to the target data identifier, and obtaining a hash value corresponding to the target data identifier.
  • Hash calculation is a calculation method for transforming an input of arbitrary length into a certain length by a hash algorithm.
  • the hash algorithm can be MD5 (Message-digest Algorithm 5) algorithm and SHA (Secure Hash Standard, security Greek standard) algorithm, etc., specifically no restrictions.
  • MD5 Message-digest Algorithm 5
  • SHA Secure Hash Standard, security Greek standard
  • Step S304 performing a remainder operation on the hash value and the preset value to obtain a target key value.
  • the target key value is used as the target key value.
  • y mod (x, b), where x represents a hash value. b indicates the preset value, and the specific value can be set according to the actual setting, such as 10000.
  • y represents the target key value.
  • Mod represents the calculation of the remainder. For example, the hash value is 3, the default value is 10000, and 3 is divided by 10000 and the remainder is 3, then 3 is the target key value.
  • Step S306 obtaining a corresponding primary cluster and a standby cluster according to the target key value.
  • the correspondence between the target key value and the primary cluster and the standby cluster may be pre-stored, and the corresponding relationship may be that the target key value directly corresponds to the primary cluster and the standby cluster, or may be an indirect correspondence.
  • the primary cluster corresponding to the target key value h is A1
  • the standby cluster is A2.
  • the step of acquiring the corresponding primary cluster and the standby cluster according to the target key value in step S306 includes:
  • Step S402 Acquire a target cluster identifier corresponding to the target key value according to the correspondence between the key value and the cluster identifier.
  • the correspondence between the key value and the cluster identifier is set in advance. As shown in Table 1, the correspondence between the key value and the cluster identifier can be stored in the form of a data table. After the target key value is obtained, the key value matching the target key value in the data table is obtained, and the cluster identifier corresponding to the key value matching the target key value is used as the target cluster identifier.
  • the correspondence between the key value and the cluster identifier may be stored in the preset storage location. After the target key value is obtained, the storage address corresponding to the preset storage location is obtained, and the key value and the cluster are obtained according to the storage address. The corresponding relationship of the logo.
  • the preset storage location may be limited according to actual needs. For example, in order to improve the speed of acquiring the target cluster identifier, the correspondence between the key value and the cluster identifier may be stored in the cache of the computer device.
  • Cluster number 1 Cluster1 2 Cluster1 3 Cluster2 4 Cluster1 5 Cluster3
  • Step S404 Acquire a primary cluster and a standby cluster corresponding to the target cluster identifier according to the correspondence between the cluster identifier and the cluster.
  • the mapping between the cluster ID and the cluster is set in advance. For example, as shown in Table 2, the primary cluster domain name and the standby cluster domain name corresponding to each cluster number are set in advance.
  • the relationship between the cluster identity and the cluster IP address can also be stored.
  • the domain name resolution request may be sent to the domain name server according to the cluster domain name information at a preset time or duration, and the correspondence between the storage cluster identifier and the cluster IP address may be obtained after receiving the IP address sent by the domain name server. In this way, each time a connection is established with the cluster, domain name resolution is not required first, which improves the timeliness of responding to data requests.
  • the computer device can continue to maintain the established network connection to continue to use the network connection to process other data requests, and then release the network connection when the preset time is, for example, one hour later. Connections, thus avoiding the delay caused by the need to create a new network connection each time a response request is sent.
  • the database domain name corresponding to the cluster identifier may be stored by the database, and the computer device may periodically read the cluster domain name information and cache it locally.
  • the data request processing method may further include the following steps:
  • Step S502 obtaining a weight corresponding to the cluster identifier.
  • the weights can be customized according to the requirements, or can be calculated by the computer device according to the performance of the cluster, for example, the capacity, so that the load of the cluster corresponds to the performance of the cluster or the number of services externally served by the cluster is adjusted according to actual needs. For example, you can calculate the capacity of a primary cluster as a percentage of the total capacity of the primary cluster, and use the percentage obtained as the weight corresponding to the cluster ID.
  • Step S504 assigning a corresponding key value to the cluster identifier according to the weight and the preset value, and obtaining a correspondence between the key value and the cluster identifier.
  • the weight is multiplied by the preset value to obtain the number of key values to be allocated by the cluster identifier, and then the corresponding key value is assigned to the cluster identifier according to the number of key values, and the correspondence between the key value and the cluster identifier is obtained.
  • the preset value is 10000
  • the weight corresponding to the A cluster is 0.25
  • the weight corresponding to the B cluster is 0.75.
  • the target data identifier is received, the target data identifier is calculated, and a hash value and a preset value of 10000 are obtained. If the remainder is 1 to 2499, the corresponding cluster is the A cluster, and the obtained remainder is obtained. If it is 2500 to 9999, the corresponding cluster is a B cluster.
  • step S208 includes:
  • Step S602 sending a second response request to the primary cluster according to the data request, where the second response request carries the target data identifier.
  • sending a second response request to the primary cluster according to the data request for example, when the data request is downloading data, sending a second response request to the primary cluster requesting that the download data identifier be the target data identifier, so that The primary cluster obtains target data based on the target data identifier and returns the target data to the computer device.
  • the second cluster response request for deleting the data identifier to the target data identifier is sent to the primary cluster, so that the primary cluster finds the target data according to the target data identifier, and deletes the target data.
  • the target data identifier and the target data are sent to the primary cluster, so that the primary cluster corresponds to the storage target data and the target data identifier.
  • Step S604 when receiving the second response information returned by the primary cluster according to the second response request, the running information is normal, and when the second response information returned by the primary cluster according to the second response request is not received, the running information is running. abnormal.
  • the primary cluster operation information is normal. For example, when the second response request is to download data, the primary cluster obtains the target data according to the target data identifier and returns the target data to the computer device. When the second response request is to delete data, the primary cluster finds the target data according to the target data identifier, and deletes the target data and returns the second data response information that is successfully deleted. When the second response request is storing data, the primary cluster returns the storage second data response information corresponding to the storage target data and the target data identifier. When the response information returned by the primary cluster according to the second response request is not received, the obtained primary cluster operation information is a running abnormality.
  • a data request processing apparatus is provided, which can be integrated into the computer device 20 described above, including:
  • the data request receiving module 702 is configured to receive a data request sent by the client, where the data request carries the target data identifier;
  • the cluster obtaining module 704 is configured to obtain a corresponding primary cluster and a standby cluster according to the target data identifier
  • the operation information obtaining module 706 is configured to obtain running information of the primary cluster, where the running information includes normal operation or abnormal operation;
  • the response information receiving module 710 is configured to receive the first data response information sent by the standby cluster according to the first response request, and return the first data response information to the client.
  • the cluster obtaining module 704 includes:
  • the calculating unit 802 is configured to perform a hash operation according to the target data identifier, and obtain a hash value corresponding to the target data identifier.
  • a remainder unit 804 configured to perform a remainder operation on the hash value and the preset value to obtain a target key value
  • the cluster obtaining unit 806 is configured to obtain a corresponding primary cluster and a standby cluster according to the target key value.
  • the cluster obtaining unit 806 includes:
  • the target identifier obtaining sub-unit 902 is configured to obtain a target cluster identifier corresponding to the target key value according to the correspondence between the key value and the cluster identifier.
  • the cluster obtaining sub-unit 904 is configured to obtain the primary cluster and the standby cluster corresponding to the target cluster identifier according to the correspondence between the cluster identifier and the cluster.
  • the data request processing apparatus further includes:
  • the weight obtaining module 1002 is configured to obtain a weight corresponding to the cluster identifier.
  • the correspondence relationship obtaining module 1004 is configured to allocate a corresponding key value to the cluster identifier according to the weight and the preset value, and obtain a correspondence between the key value and the cluster identifier.
  • the operation information obtaining module 706 includes:
  • the second response request sending unit 1102 is configured to send a second response request to the primary cluster according to the data request, where the second response request carries the target data identifier.
  • the operation information obtaining unit 1104 is configured to: when receiving the second response information returned by the primary cluster according to the second response request, the running information is normal, when the second response information returned by the primary cluster according to the second response request is not received.
  • the running information is abnormal.
  • the various modules in the above data request processing device may be implemented in whole or in part by software, hardware, and combinations thereof.
  • Each of the above modules may be embedded in or independent of the processor in the computer device, or may be stored in a memory in the computer device in a software form, so that the processor invokes the operations corresponding to the above modules.
  • the computer device may be a server connected to a processor, a non-volatile storage medium, an internal memory, and a network interface through a system connection bus.
  • the non-volatile storage medium can be a non-transitory computer readable storage medium.
  • the non-volatile storage medium of the computer device can store an operating system and computer readable instructions that, when executed, cause the processor to perform a data request processing method.
  • the processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device.
  • the internal memory can store computer readable instructions that, when executed by the processor, cause the processor to perform a data request processing method.
  • the network interface of the computer device is used for network communication, such as receiving a data request, sending a data response request, and the like.
  • FIG. 12 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation of the computer device to which the solution of the present application is applied.
  • the specific computer device may It includes more or fewer components than those shown in the figures, or some components are combined, or have different component arrangements.
  • the data request processing apparatus can be implemented in the form of a computer readable instruction that can be run on a computer device as shown in FIG.
  • the dysfunctional storage medium can store various program modules constituting the data request processing device, such as the data request receiving module 702, the cluster obtaining module 704, the running information acquiring module 706, the first response request sending module 708, and the response information receiving device in FIG. Module 710.
  • Each of the program modules includes computer readable instructions for causing the computer device to perform the steps in the data request processing method of various embodiments of the present application described in this specification, for example, the computer device can The data request receiving module 702 in the data request processing device shown in FIG.
  • the cluster 7 receives the data request sent by the client, and the data request carries the target data identifier, and the cluster obtaining module 704 obtains the corresponding primary cluster and the backup according to the target data identifier.
  • the cluster obtains the running information of the primary cluster by using the running information obtaining module 706.
  • the running information includes a normal operation or an abnormal operation.
  • the first response request sending module 708 sends a first response request to the standby cluster according to the data request, where the first response request carries The target data identifier is received by the response information receiving module 710, and the first data response information sent by the standby cluster according to the first response request is returned to the client.
  • a computer device comprising a memory and one or more processors, the memory storing computer readable instructions, the computer readable instructions being executed by the processor, causing the one or more processors to perform the steps of: receiving a Data request, the data request carries the target data identifier; the corresponding primary cluster and the standby cluster are obtained according to the target data identifier; the running information of the primary cluster is obtained, and the running information includes normal operation or abnormal operation; if the primary cluster runs abnormally, according to the data request
  • the standby cluster sends a first response request, where the first response request carries the target data identifier; and the first data response information sent by the standby cluster according to the first response request and the first data response information is returned to the client.
  • One or more non-transitory computer readable storage mediums storing computer readable instructions, when executed by one or more processors, cause one or more processors to perform the steps of: receiving a client to send The data request carries the target data identifier; the corresponding primary cluster and the standby cluster are obtained according to the target data identifier; the running information of the primary cluster is obtained, and the running information includes normal operation or abnormal operation; if the primary cluster runs abnormally, according to the data request Sending a first response request to the standby cluster, where the first response request carries the target data identifier; and receiving the first data response information sent by the standby cluster according to the first response request, and returning the first data response information to the client.
  • Non-volatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM) or external cache memory.
  • RAM is available in a variety of formats, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization chain.
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDRSDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • Synchlink DRAM SLDRAM
  • Memory Bus Radbus
  • RDRAM Direct RAM
  • DRAM Direct Memory Bus Dynamic RAM
  • RDRAM Memory Bus Dynamic RAM

Abstract

A method for processing a data request, comprising: receiving a data request sent by a client, the data request carrying a target data identifier; obtaining a corresponding master cluster and a corresponding standby cluster according to the target data identifier; obtaining running information of the master cluster, the running information comprising running normally or running abnormally; if the master cluster runs abnormally, sending a first response request to the standby cluster according to the data request, the first response request carrying the target data identifier; and receiving first data response information sent by the standby cluster according to the first response request, and sending back the first data response information to the client.

Description

数据请求处理方法、装置、计算机设备和存储介质Data request processing method, device, computer device and storage medium
相关申请的交叉引用Cross-reference to related applications
本申请要求于2017年11月14日提交中国专利局,申请号为2017111222220,申请名称为“数据请求处理方法、装置、计算机设备和存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese Patent Application entitled "Data Request Processing Method, Apparatus, Computer Equipment, and Storage Media", which is filed on November 14, 2017, and the application number is 2017111222220, the entire contents of which are incorporated by reference. Combined in this application.
技术领域Technical field
本申请涉及数据请求处理方法、装置、计算机设备和存储介质。The present application relates to a data request processing method, apparatus, computer device, and storage medium.
背景技术Background technique
随着科学技术的发展,数据的存储和管理已经成为越来越受关注的热点。为了应对不断增长的数据存储以及管理的需求,可以通过集群进行数据存储和管理服务。集群是指多台服务器通过网络通常是高速局域网互联起来组成的一个整体,统一向外部客户端提供服务,对外看起来就像是一台服务器。然而,发明人意识到,当集群容量不够需要新增集群时,由于新的集群的域名与原集群的域名不同,因此需要将原来集群的数据迁移到新集群中,工作量大,而且当主集群发生故障,需要启用备集群进行服务时,需要人工更改域名解析方案,时效性差。With the development of science and technology, the storage and management of data has become a hot spot of increasing concern. In order to cope with the growing demand for data storage and management, data storage and management services can be performed through clusters. A cluster is a whole of multiple servers connected by a network, usually a high-speed LAN. It provides services to external clients in a unified manner, and it looks like a server. However, the inventor realized that when the cluster capacity is insufficient and a new cluster is needed, since the domain name of the new cluster is different from the domain name of the original cluster, the data of the original cluster needs to be migrated to the new cluster, the workload is large, and the main cluster is When a fault occurs and the standby cluster needs to be enabled for service, the domain name resolution scheme needs to be manually changed, and the timeliness is poor.
发明内容Summary of the invention
根据本申请公开的各种实施例,提供一种数据请求处理方法、装置、计算机设备和存储介质。According to various embodiments disclosed herein, a data request processing method, apparatus, computer device, and storage medium are provided.
一种数据请求处理方法,包括:A data request processing method includes:
接收客户端发送的数据请求,所述数据请求携带有目标数据标识;根据所述目标数据标识得到对应的主集群以及备集群;Receiving a data request sent by the client, where the data request carries a target data identifier; and obtaining a corresponding primary cluster and a standby cluster according to the target data identifier;
获取所述主集群的运行信息,所述运行信息包括运行正常或运行异常;Obtaining operation information of the primary cluster, where the running information includes a normal operation or an abnormal operation;
若所述主集群运行异常,根据所述数据请求向所述备集群发送第一响应请求,所述第一响应请求携带有所述目标数据标识;及And if the primary cluster runs abnormally, sending a first response request to the standby cluster according to the data request, where the first response request carries the target data identifier;
接收所述备集群根据所述第一响应请求发送的第一数据响应信息并将所述第一数据响应 信息返回给所述客户端。Receiving, by the standby cluster, first data response information sent according to the first response request, and returning the first data response information to the client.
一种数据请求处理装置,包括:A data request processing apparatus includes:
数据请求接收模块,用于接收客户端发送的数据请求,所述数据请求携带有目标数据标识;a data request receiving module, configured to receive a data request sent by the client, where the data request carries a target data identifier;
集群得到模块,用于根据所述目标数据标识得到对应的主集群以及备集群;运行信息获取模块,用于获取所述主集群的运行信息,所述运行信息包括运行正常或运行异常;The cluster obtaining module is configured to obtain a corresponding primary cluster and a standby cluster according to the target data identifier, and the operation information obtaining module is configured to obtain the running information of the primary cluster, where the running information includes normal operation or abnormal operation;
第一响应请求发送模块,若所述主集群运行异常,根据所述数据请求向所述备集群发送第一响应请求,所述第一响应请求携带有所述目标数据标识;及a first response request sending module, if the primary cluster runs abnormally, sending a first response request to the standby cluster according to the data request, where the first response request carries the target data identifier;
响应信息接收模块,用于接收所述备集群根据所述第一响应请求发送的第一数据响应信息并将所述第一数据响应信息返回给所述客户端。The response information receiving module is configured to receive first data response information sent by the standby cluster according to the first response request, and return the first data response information to the client.
一种计算机设备,包括存储器和一个或多个处理器,存储器中存储有计算机可读指令,计算机可读指令被处理器执行时实现本申请任意一个实施例中提供的数据请求处理方法的步骤。。A computer apparatus comprising a memory and one or more processors having stored therein computer readable instructions that, when executed by a processor, implement the steps of a data request processing method provided in any one of the embodiments of the present application. .
一个或多个存储有计算机可读指令的非易失性计算机可读存储介质,计算机可读指令被一个或多个处理器执行时,使得一个或多个处理器实现本申请任意一个实施例中提供的数据请求处理方法的步骤。One or more non-transitory computer readable storage mediums storing computer readable instructions, when executed by one or more processors, cause one or more processors to be implemented in any one embodiment of the present application The steps of the data request processing method provided.
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征和优点将从说明书、附图以及权利要求书变得明显。Details of one or more embodiments of the present application are set forth in the accompanying drawings and description below. Other features and advantages of the present invention will be apparent from the description, drawings and claims.
附图说明DRAWINGS
为了更清楚地说明本申请实施例中的技术方案,下面将对实施例中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其它的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings to be used in the embodiments will be briefly described below. Obviously, the drawings in the following description are only some embodiments of the present application, Those skilled in the art can also obtain other drawings based on these drawings without any creative work.
图1为根据一个或多个实施例中提供的数据请求处理方法的应用环境图。1 is an application environment diagram of a data request processing method provided in accordance with one or more embodiments.
图2为根据一个或多个实施例中数据请求处理方法的流程图。2 is a flow diagram of a data request processing method in accordance with one or more embodiments.
图3为根据一个或多个实施例中根据目标数据标识得到对应的主集群以及备集群的流程图。FIG. 3 is a flowchart of obtaining a corresponding primary cluster and a standby cluster according to target data identification according to one or more embodiments.
图4为根据一个或多个实施例中根据目标键值获取对应的主集群以及备集群的流程图。4 is a flow diagram of obtaining a corresponding primary cluster and standby cluster based on target key values in accordance with one or more embodiments.
图5为另一个实施例中数据请求处理方法的流程图。FIG. 5 is a flow chart of a data request processing method in another embodiment.
图6为根据一个或多个实施例中获取主集群的运行信息的流程图。6 is a flow diagram of obtaining operational information for a primary cluster in accordance with one or more embodiments.
图7为根据一个或多个实施例中数据请求处理装置的框图。FIG. 7 is a block diagram of a data request processing apparatus in accordance with one or more embodiments.
图8为根据一个或多个实施例中集群得到模块的框图。FIG. 8 is a block diagram of a cluster get module in accordance with one or more embodiments.
图9为根据一个或多个实施例中集群得到单元的框图。9 is a block diagram of a clustering unit in accordance with one or more embodiments.
图10为另一个实施例中数据请求处理装置的框图。Figure 10 is a block diagram of a data request processing apparatus in another embodiment.
图11为根据一个或多个实施例中运行信息获取模块的框图。11 is a block diagram of a run information acquisition module in accordance with one or more embodiments.
图12为根据一个或多个实施例中计算机设备的框图。Figure 12 is a block diagram of a computer device in accordance with one or more embodiments.
具体实施方式Detailed ways
为了使本申请的技术方案及优点更加清楚明白,以下结合附图及实施例,对本申请进行进一步详细说明。应当理解,此处所描述的具体实施例仅仅用以解释本申请,并不用于限定本申请。In order to make the technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the application and are not intended to be limiting.
可以理解,本申请所使用的术语“第一”、“第二”等可在本文中用于描述各种元件,但除非特别说明,这些元件不受这些术语限制。这些术语仅用于将第一个元件与另一个元件区分。举例来说,在不脱离本申请的范围的情况下,可以将第一主集群称为第二主集群,且类似地,可将第二主集群称为第一主集群。It will be understood that the terms "first," "second," and the like, as used herein, may be used to describe various elements, and the elements are not limited by these terms unless specifically stated. These terms are only used to distinguish one element from another. For example, the first primary cluster may be referred to as a second primary cluster without departing from the scope of the present application, and similarly, the second primary cluster may be referred to as a first primary cluster.
图1为一个实施例中提供的数据请求处理方法的应用环境图,如图1所示,在该实施环境中,如上图所示,包括客户端10、计算机设备20、第一主集群31、第一主集群31对应的第一备集群32、第二主集群41及第二主集群41对应的第二备集群42。可以通过网络进行连接。主集群与备集群一般分布在不同城市,以在当主集群发生故障时,由备集群对业务进行响应。第一主集群31以及第二主集群的域名是不一样的,但是在对外服务时,可以设置一个相同的域名,这个相同的域名的域名解析结果指向计算机设备20的IP地址。当客户端发出数据请求例如请求下载某个文件时,由计算机设备20接收数据请求,根据数据请求中的目标数据标识得到对应的主集群域名,然后根据主集群域名以及数据请求向主集群发送响应请求,使对应的主集群进行响应并返回响应信息,并将响应信息发送给客户端。若主集群发生故障没有响应,代理服务器根据备集群域名以及数据请求向备集群发送响应请求,接收备集群返回的响应信息并将响应信息发送给客户端。FIG. 1 is an application environment diagram of a data request processing method provided in an embodiment. As shown in FIG. 1 , in the implementation environment, as shown in the foregoing figure, the client 10, the computer device 20, and the first main cluster 31 are included. The first standby cluster 32 corresponding to the first primary cluster 31, the second primary cluster 41, and the second standby cluster 42 corresponding to the second primary cluster 41. It can be connected via the network. The primary cluster and the standby cluster are generally distributed in different cities, so that when the primary cluster fails, the standby cluster responds to the service. The domain names of the first primary cluster 31 and the second primary cluster are different, but in the external service, an identical domain name may be set, and the domain name resolution result of the same domain name points to the IP address of the computer device 20. When the client issues a data request, for example, requesting to download a file, the computer device 20 receives the data request, obtains the corresponding primary cluster domain name according to the target data identifier in the data request, and then sends a response to the primary cluster according to the primary cluster domain name and the data request. The request causes the corresponding primary cluster to respond and return a response message, and sends the response information to the client. If the primary cluster fails to respond, the proxy server sends a response request to the standby cluster according to the standby cluster domain name and the data request, and receives the response information returned by the standby cluster and sends the response information to the client.
计算机设备20可以是独立的物理服务器或终端,也可以是多个物理服务器构成的服务器集群,可以是提供云服务器、云数据库、云存储和CDN等基础云计算服务的云服务器。客户端可以是智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表、服务器等,但并不局限于此。客户端10与计算机设备20、计算机设备20与第一主集群31、第一备集群 32、第二主集群41、第二备集群42、第一主集群31与第一备集群32以及第二主集群41与第二备集群42之间可以通过网络进行连接。集群可以有一个或多个,例如可以有10个,本申请实施例对此不作限制。The computer device 20 may be a stand-alone physical server or terminal, or may be a server cluster composed of multiple physical servers, and may be a cloud server that provides basic cloud computing services such as a cloud server, a cloud database, a cloud storage, and a CDN. The client can be a smartphone, a tablet, a laptop, a desktop computer, a smart speaker, a smart watch, a server, etc., but is not limited thereto. Client 10 and computer device 20, computer device 20 and first primary cluster 31, first standby cluster 32, second primary cluster 41, second standby cluster 42, first primary cluster 31 and first standby cluster 32, and second The primary cluster 41 and the second standby cluster 42 can be connected through a network. There may be one or more clusters, for example, there may be 10, which is not limited by the embodiment of the present application.
如图2所示,在一些实施例中,提出了一种数据请求处理方法,该数据请求处理方法可以应用于上述的计算机设备20中,具体可以包括以下步骤:As shown in FIG. 2, in some embodiments, a data request processing method is provided, which may be applied to the computer device 20, and may specifically include the following steps:
步骤S202,接收客户端发送的数据请求,数据请求携带有目标数据标识。Step S202, receiving a data request sent by the client, where the data request carries the target data identifier.
数据请求可以包括请求下载集群中的数据、请求删除集群中的数据或者请求向集群上传数据中的一个或多个。当为向集群上传数据时,数据请求中还携带有要上传的数据。目标数据标识可以用数据以及文字的至少一个进行表示。一个或多个集群是通过统一的访问地址对外进行服务的。客户端接收该访问地址例如域名或者计算机设备的IP地址,根据访问地址以向计算机设备发送数据请求。当输入的访问地址为域名时,数据请求经过DNS(Domain Name System,域名系统)解析后得到计算设备20的IP地址,以将数据请求路由到计算机设备20中当输入的访问地址为IP地址时,客户端根据IP地址将数据请求发送到计算机设备20中。The data request may include requesting to download data in the cluster, requesting deletion of data in the cluster, or requesting one or more of uploading data to the cluster. When uploading data to the cluster, the data request also carries the data to be uploaded. The target data identifier can be represented by at least one of data and text. One or more clusters are served externally through a uniform access address. The client receives the access address, such as a domain name or an IP address of a computer device, and sends a data request to the computer device based on the access address. When the input access address is a domain name, the data request is parsed by a DNS (Domain Name System) to obtain an IP address of the computing device 20 to route the data request to the computer device 20 when the input access address is an IP address. The client sends a data request to the computer device 20 based on the IP address.
步骤S204,根据目标数据标识得到对应的主集群以及备集群。Step S204, obtaining a corresponding primary cluster and a standby cluster according to the target data identifier.
可以预先设置数据标识与集群的对应关系,该对应关系可以是直接对应关系,例如,数据标识a对应b集群。也可以是间接对应关系。例如,可以设置计算规则以及数值与集群的对应关系,当得到数据标识后,根据预先设置的计算规则对数据标识进行计算,得到相应的数值再根据数值与集群的对应关系得到对应的主集群以及备集群。The correspondence between the data identifier and the cluster may be set in advance, and the correspondence may be a direct correspondence. For example, the data identifier a corresponds to the b cluster. It can also be an indirect correspondence. For example, the calculation rule and the correspondence between the value and the cluster may be set. After the data identifier is obtained, the data identifier is calculated according to the preset calculation rule, and the corresponding value is obtained, and the corresponding primary cluster is obtained according to the correspondence between the value and the cluster. Standby cluster.
步骤S206,获取主集群的运行信息,运行信息包括运行正常或运行异常。In step S206, the running information of the primary cluster is obtained, and the running information includes a normal operation or an abnormal operation.
主集群的运行信息可以是主动获取的,也可以是主集群自动发送的。例如,主集群可以定时向计算机设备发送心跳信息,若计算机在一定时间内未收到该集群的心跳信息,则表示主集群运行异常。在一些实施例中,为了能够在主集群发生故障恢复正常后能够及时地进行工作,接收到数据请求后,可以将数据请求发送到到主集群中,若没有接收到主集群的响应,则表示主集群运行异常。The running information of the primary cluster can be obtained actively or automatically by the primary cluster. For example, the primary cluster can periodically send heartbeat information to the computer device. If the computer does not receive the heartbeat information of the cluster within a certain period of time, it indicates that the primary cluster is abnormal. In some embodiments, in order to be able to work in time after the primary cluster fails to recover normally, after receiving the data request, the data request may be sent to the primary cluster, and if the response of the primary cluster is not received, The primary cluster is running abnormally.
步骤S208,若主集群运行异常,根据数据请求向备集群发送第一响应请求,第一响应请求携带有目标数据标识。Step S208: If the primary cluster runs abnormally, send a first response request to the standby cluster according to the data request, where the first response request carries the target data identifier.
主集群异常指主集群不能进行响应,若主集群运行异常,根据数据请求向备集群发送第一响应请求。例如,当数据请求为下载数据时,则向备集群发送请求下载数据标识为目标数据标识的第一响应请求,以使备集群根据目标数据标识得到目标数据,并将目标数据返回给计算机设备。当数据请求为删除数据时,向备集群发送请求删除数据标识为目标数据标识的 第一响应请求,以使备集群根据目标数据标识找到目标数据,并删除该目标数据。当数据请求为存储数据时,向备集群发送该目标数据标识以及目标数据,使备集群对应存储目标数据以及目标数据标识。The primary cluster abnormality means that the primary cluster cannot respond. If the primary cluster runs abnormally, the first response request is sent to the standby cluster according to the data request. For example, when the data request is downloading data, the first response request for requesting the download data identifier to be the target data identifier is sent to the standby cluster, so that the standby cluster obtains the target data according to the target data identifier, and returns the target data to the computer device. When the data request is to delete data, the first response request for deleting the data identifier to the target data identifier is sent to the standby cluster, so that the standby cluster finds the target data according to the target data identifier, and deletes the target data. When the data request is to store data, the target data identifier and the target data are sent to the standby cluster, so that the standby cluster corresponds to the storage target data and the target data identifier.
在一些实施例中,若主集群运行正常,则根据数据请求向主集群发送第二响应请求,使主集群进行响应并返回第二响应信息,计算机设备将第二响应信息返回给客户端。In some embodiments, if the primary cluster is running normally, the second response request is sent to the primary cluster according to the data request, the primary cluster responds and returns the second response information, and the computer device returns the second response information to the client.
在一些实施例中,主集群和备集群可以通过异地同步程序进行同步,异地同步程序读取主集群操作记录,根据主集群操作记录将主集群的操作增量同步到备集群。In some embodiments, the primary cluster and the standby cluster can be synchronized by the remote synchronization program, and the remote synchronization program reads the primary cluster operation record, and synchronizes the operation of the primary cluster to the standby cluster according to the primary cluster operation record.
步骤S210,接收备集群根据第一响应请求发送的第一数据响应信息并将第一数据响应信息返回给客户端。Step S210: Receive first data response information sent by the standby cluster according to the first response request, and return the first data response information to the client.
第一数据响应信息为备集群根据第一响应请求返回的信息。当第一响应请求为下载数据时,备集群根据目标数据标识得到目标数据,并将目标数据返回给计算机设备。当第一响应请求为删除数据时,备集群根据目标数据标识找到目标数据,删除该目标数据后返回删除成功的第一数据响应信息。当第一响应请求为存储数据时,备集群对应存储目标数据以及目标数据标识后返回存储成功的第一数据响应信息。The first data response information is information returned by the standby cluster according to the first response request. When the first response request is to download data, the standby cluster obtains the target data according to the target data identifier, and returns the target data to the computer device. When the first response request is to delete data, the standby cluster finds the target data according to the target data identifier, and deletes the target data and returns the first data response information that is successfully deleted. When the first response request is to store data, the standby cluster returns the storage first data response information corresponding to the storage target data and the target data identifier.
在一些实施例中,当第一响应请求为下载数据时,备集群返回要下载的数据,当第一响应请求为上传数据或者删除数据时,为保持主集群与备集群的一致性,备集群返回请求错误的第一响应信息。In some embodiments, when the first response request is to download data, the standby cluster returns data to be downloaded. When the first response request is uploading data or deleting data, in order to maintain consistency between the primary cluster and the standby cluster, the standby cluster is configured. Returns the first response message requesting an error.
上述数据请求处理方法,在接收到客户端发送的数据请求时,根据目标数据标识得到对应的主集群以及备集群,并在主集群运行异常时,向备集群发送第一响应请求,接收备集群根据第一响应请求发送的第一数据响应信息并将第一数据响应信息返回给客户端。由于接收到数据请求时根据目标数据标识得到对应的主集群以及备集群,并在主集群运行异常时根据数据请求请求对应的备集群进行响应,因此可以通过统一的访问地址对外进行服务,新增集群时无需对数据进行迁移,减少了迁移工作量而且在主集群发生故障时能够及时通过备集群进行响应。When receiving the data request sent by the client, the data request processing method obtains the corresponding primary cluster and the standby cluster according to the target data identifier, and sends a first response request to the standby cluster when the primary cluster runs abnormally, and receives the standby cluster. And transmitting the first data response information according to the first response request to the client. When the data request is received, the corresponding primary cluster and the standby cluster are obtained according to the target data identifier, and when the primary cluster runs abnormally, the corresponding standby cluster responds according to the data request, so that the service can be externally accessed through the unified access address. The cluster does not need to migrate data, which reduces the workload of the migration and can respond to the standby cluster in time when the primary cluster fails.
在一些实施例中,如图3所示,步骤S204即根据目标数据标识得到对应的主集群以及备集群包括:In some embodiments, as shown in FIG. 3, step S204 is to obtain a corresponding primary cluster and a standby cluster according to the target data identifier, including:
步骤S302,根据目标数据标识进行哈希运算,得到目标数据标识对应的哈希值。Step S302, performing a hash operation according to the target data identifier, and obtaining a hash value corresponding to the target data identifier.
哈希计算为把任意长度的输入通过散列算法变换成一定长度的输出的计算方法,散列算法可以为MD5(Message-digest Algorithm5,信息摘要算法5)算法和SHA(Secure Hash Standard、安全哈希标准)算法等,具体不做限制。当得到目标数据标识后,利用预设的算法 对目标数据标识进行计算,得到哈希值。Hash calculation is a calculation method for transforming an input of arbitrary length into a certain length by a hash algorithm. The hash algorithm can be MD5 (Message-digest Algorithm 5) algorithm and SHA (Secure Hash Standard, security Greek standard) algorithm, etc., specifically no restrictions. After the target data identifier is obtained, the target data identifier is calculated by using a preset algorithm to obtain a hash value.
步骤S304,对哈希值以及预设的数值进行取余运算,得到目标键值。Step S304, performing a remainder operation on the hash value and the preset value to obtain a target key value.
得到哈希值后,将哈希值除以预设的数值得到的余数作为目标键值。以公式表示如下:y=mod(x,b),其中,x表示哈希值。b表示预设的数值,具体数值可以根据实际设置,如10000。y表示目标键值。mod表示进行取余计算。例如,哈希值为3,预设的数值为10000,3除以10000余数为3,则3为目标键值。After the hash value is obtained, the remainder obtained by dividing the hash value by the preset value is used as the target key value. Expressed as follows: y = mod (x, b), where x represents a hash value. b indicates the preset value, and the specific value can be set according to the actual setting, such as 10000. y represents the target key value. Mod represents the calculation of the remainder. For example, the hash value is 3, the default value is 10000, and 3 is divided by 10000 and the remainder is 3, then 3 is the target key value.
步骤S306,根据目标键值获取对应的主集群以及备集群。Step S306, obtaining a corresponding primary cluster and a standby cluster according to the target key value.
可以预先存储目标键值与主集群以及备集群的对应关系,该对应关系可以是目标键值与主集群以及备集群直接对应,也可以是间接对应。例如,目标键值h对应的主集群为A1,备集群为A2。The correspondence between the target key value and the primary cluster and the standby cluster may be pre-stored, and the corresponding relationship may be that the target key value directly corresponds to the primary cluster and the standby cluster, or may be an indirect correspondence. For example, the primary cluster corresponding to the target key value h is A1, and the standby cluster is A2.
在一些实施例中,如图4所示,步骤S306即根据目标键值获取对应的主集群以及备集群的步骤包括:In some embodiments, as shown in FIG. 4, the step of acquiring the corresponding primary cluster and the standby cluster according to the target key value in step S306 includes:
步骤S402,根据键值与集群标识的对应关系获取目标键值对应的目标集群标识。Step S402: Acquire a target cluster identifier corresponding to the target key value according to the correspondence between the key value and the cluster identifier.
预先设置了键值与集群标识的对应关系。如表1所示,可以以数据表的形式存储键值与集群标识的对应关系。当得到目标键值后,获取数据表中与目标键值匹配的键值,将与目标键值匹配的键值对应的集群标识作为目标集群标识。The correspondence between the key value and the cluster identifier is set in advance. As shown in Table 1, the correspondence between the key value and the cluster identifier can be stored in the form of a data table. After the target key value is obtained, the key value matching the target key value in the data table is obtained, and the cluster identifier corresponding to the key value matching the target key value is used as the target cluster identifier.
在一些实施例中,可以将键值与集群标识的对应关系存储在预设存储位置中,当得到目标键值后,获取该预设存储位置对应的存储地址,根据存储地址得到键值与集群标识的对应关系。预设存储位置可以根据实际需要进行限制,例如,为了提高获取目标集群标识的速度,可以将键值与集群标识的对应关系存储在计算机设备的高速缓冲存储器中。In some embodiments, the correspondence between the key value and the cluster identifier may be stored in the preset storage location. After the target key value is obtained, the storage address corresponding to the preset storage location is obtained, and the key value and the cluster are obtained according to the storage address. The corresponding relationship of the logo. The preset storage location may be limited according to actual needs. For example, in order to improve the speed of acquiring the target cluster identifier, the correspondence between the key value and the cluster identifier may be stored in the cache of the computer device.
表1:Table 1:
键值Key value 集群编号Cluster number
11 cluster1Cluster1
22 cluster1Cluster1
33 Cluster2Cluster2
44 cluster1Cluster1
55 Cluster3Cluster3
步骤S404,根据集群标识与集群的对应关系获取目标集群标识对应的主集群以及备集群。Step S404: Acquire a primary cluster and a standby cluster corresponding to the target cluster identifier according to the correspondence between the cluster identifier and the cluster.
预先设置了集群标识与集群的对应关系,例如可如表2所示,预先设置了每个集群编号对应的主集群域名以及备集群域名。The mapping between the cluster ID and the cluster is set in advance. For example, as shown in Table 2, the primary cluster domain name and the standby cluster domain name corresponding to each cluster number are set in advance.
在一些实施例中,还可以存储集群标识与集群ip地址的关系。例如可以在预设时间或时长根据集群域名信息向域名服务器发送域名解析请求,接收到域名服务器发送的IP地址后,存储集群标识与集群ip地址的对应关系。这样,每次与集群建立连接时无需先进行域名解析,提高了对数据请求进行响应的及时性。In some embodiments, the relationship between the cluster identity and the cluster IP address can also be stored. For example, the domain name resolution request may be sent to the domain name server according to the cluster domain name information at a preset time or duration, and the correspondence between the storage cluster identifier and the cluster IP address may be obtained after receiving the IP address sent by the domain name server. In this way, each time a connection is established with the cluster, domain name resolution is not required first, which improves the timeliness of responding to data requests.
在一些实施例中,计算机设备与集群建立连接并获取响应信息后,可以继续保持建立的网络连接,以继续利用该网络连接处理其他数据请求,当网络连接达到预设时长例如一个小时后再释放连接,因此避免了每次发送响应请求均需要新建网络连接造成的延时。In some embodiments, after establishing a connection with the cluster and obtaining response information, the computer device can continue to maintain the established network connection to continue to use the network connection to process other data requests, and then release the network connection when the preset time is, for example, one hour later. Connections, thus avoiding the delay caused by the need to create a new network connection each time a response request is sent.
在一些实施例中,可以利用数据库持久化存储集群标识对应的集群域名,计算机设备可以定时读取集群域名信息并缓存到本地。In some embodiments, the database domain name corresponding to the cluster identifier may be stored by the database, and the computer device may periodically read the cluster domain name information and cache it locally.
表2:Table 2:
Figure PCTCN2018077132-appb-000001
Figure PCTCN2018077132-appb-000001
在一些实施例中,如图5所示,数据请求处理方法还可以包括以下步骤:In some embodiments, as shown in FIG. 5, the data request processing method may further include the following steps:
步骤S502,获取集群标识对应的权重。Step S502, obtaining a weight corresponding to the cluster identifier.
权重具体可以根据需求进行自定义设置,也可以由计算机设备根据集群的性能例如容量进行计算得到,以使集群的负载与集群性能对应或者根据实际需要调整集群对外进行服务的数量。例如,可以计算一个主集群的容量占全部主集群容量的百分比,将得到的百分比作为集群标识对应的权重。The weights can be customized according to the requirements, or can be calculated by the computer device according to the performance of the cluster, for example, the capacity, so that the load of the cluster corresponds to the performance of the cluster or the number of services externally served by the cluster is adjusted according to actual needs. For example, you can calculate the capacity of a primary cluster as a percentage of the total capacity of the primary cluster, and use the percentage obtained as the weight corresponding to the cluster ID.
步骤S504,根据权重以及预设的数值为集群标识分配对应的键值,得到键值与集群标识的对应关系。Step S504, assigning a corresponding key value to the cluster identifier according to the weight and the preset value, and obtaining a correspondence between the key value and the cluster identifier.
得到权重后,将权重与预设的数值相乘,得到集群标识需要分配的键值数量,然后根据键值数量为集群标识分配对应的键值,得到键值与集群标识的对应关系。After the weight is obtained, the weight is multiplied by the preset value to obtain the number of key values to be allocated by the cluster identifier, and then the corresponding key value is assigned to the cluster identifier according to the number of key values, and the correspondence between the key value and the cluster identifier is obtained.
例如,预设的数值为10000,A集群对应的权重为0.25,B集群对应的权重为0.75,按照 权重以及总数量将键值分配给各个集群,则A集群对应的键值数量为0.25*10000=2500,故可以将键值0~2499分配给A集群,B集群对应的键值数量为0.75*10000=7500,故将键值2500~9999分配给B集群。当接收到目标数据标识时,对目标数据标识进行计算,得到一个哈希值与预设的数值10000取余,得到的余数若为1~2499,则对应的集群为A集群,得到的余数若为2500~9999,则对应的集群为B集群。For example, the preset value is 10000, the weight corresponding to the A cluster is 0.25, and the weight corresponding to the B cluster is 0.75. The key value is assigned to each cluster according to the weight and the total number, and the number of keys corresponding to the A cluster is 0.25*10000. =2500, so the key value 0~2499 can be assigned to the A cluster. The number of keys corresponding to the B cluster is 0.75*10000=7500, so the key value 2500-9999 is assigned to the B cluster. When the target data identifier is received, the target data identifier is calculated, and a hash value and a preset value of 10000 are obtained. If the remainder is 1 to 2499, the corresponding cluster is the A cluster, and the obtained remainder is obtained. If it is 2500 to 9999, the corresponding cluster is a B cluster.
在一些实施例中,如图6所示,步骤S208即获取主集群的运行信息的步骤包括:In some embodiments, as shown in FIG. 6, the step of acquiring the operation information of the primary cluster in step S208 includes:
步骤S602,根据数据请求向主集群发送第二响应请求,第二响应请求携带有目标数据标识。Step S602, sending a second response request to the primary cluster according to the data request, where the second response request carries the target data identifier.
当接收到数据请求后,根据数据请求向主集群发送第二响应请求,例如,当数据请求为下载数据时,则向主集群发送请求下载数据标识为目标数据标识的第二响应请求,以使主集群根据目标数据标识得到目标数据,并将目标数据返回给计算机设备。当数据请求为删除数据时,向主集群发送请求删除数据标识为目标数据标识的第二响应请求,以使主集群根据目标数据标识找到目标数据,并删除该目标数据。当数据请求为存储数据时,向主集群发送该目标数据标识以及目标数据,使主集群对应存储目标数据以及目标数据标识。After receiving the data request, sending a second response request to the primary cluster according to the data request, for example, when the data request is downloading data, sending a second response request to the primary cluster requesting that the download data identifier be the target data identifier, so that The primary cluster obtains target data based on the target data identifier and returns the target data to the computer device. When the data request is to delete data, the second cluster response request for deleting the data identifier to the target data identifier is sent to the primary cluster, so that the primary cluster finds the target data according to the target data identifier, and deletes the target data. When the data request is to store data, the target data identifier and the target data are sent to the primary cluster, so that the primary cluster corresponds to the storage target data and the target data identifier.
步骤S604,当接收到主集群根据第二响应请求返回的第二响应信息时,运行信息为运行正常,当未接收到主集群根据第二响应请求返回的第二响应信息时,运行信息为运行异常。Step S604, when receiving the second response information returned by the primary cluster according to the second response request, the running information is normal, and when the second response information returned by the primary cluster according to the second response request is not received, the running information is running. abnormal.
发送第二响应请求后,若接收到主集群根据第二响应请求返回的第二响应信息,则主集群运行信息为运行正常。例如,当第二响应请求为下载数据时,主集群根据目标数据标识得到目标数据,并将目标数据返回给计算机设备。当第二响应请求为删除数据时,主集群根据目标数据标识找到目标数据,删除该目标数据后返回删除成功的第二数据响应信息。当第二响应请求为存储数据时,主集群对应存储目标数据以及目标数据标识后返回存储成功的第二数据响应信息。当未接收到主集群根据第二响应请求返回的响应信息时,则得到的主集群运行信息为运行异常。After the second response request is sent, if the second response information returned by the primary cluster according to the second response request is received, the primary cluster operation information is normal. For example, when the second response request is to download data, the primary cluster obtains the target data according to the target data identifier and returns the target data to the computer device. When the second response request is to delete data, the primary cluster finds the target data according to the target data identifier, and deletes the target data and returns the second data response information that is successfully deleted. When the second response request is storing data, the primary cluster returns the storage second data response information corresponding to the storage target data and the target data identifier. When the response information returned by the primary cluster according to the second response request is not received, the obtained primary cluster operation information is a running abnormality.
应该理解的是,虽然上述的流程图中的各个步骤按照箭头的指示依次显示,但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明,这些步骤的执行并没有严格的顺序限制,这些步骤可以以其它的顺序执行。而且,上述流程图的至少一部分步骤可以包括多个子步骤或者多个阶段,这些子步骤或者阶段并不必然是在同一时刻执行完成,而是可以在不同的时刻执行,这些子步骤或者阶段的执行顺序也不必然是依次进行,而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。It should be understood that although the various steps in the above-described flowcharts are sequentially displayed as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Except as explicitly stated herein, the execution of these steps is not strictly limited, and the steps may be performed in other orders. Moreover, at least a portion of the steps of the above-described flowcharts may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be executed at different times, the execution of these sub-steps or stages The order is also not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of the other steps.
如图7所示,在一些实施例中,提供了一种数据请求处理装置,该数据请求处理装置可 以集成于上述的计算机设备20中,包括:As shown in FIG. 7, in some embodiments, a data request processing apparatus is provided, which can be integrated into the computer device 20 described above, including:
数据请求接收模块702,用于接收客户端发送的数据请求,数据请求携带有目标数据标识;The data request receiving module 702 is configured to receive a data request sent by the client, where the data request carries the target data identifier;
集群得到模块704,用于根据目标数据标识得到对应的主集群以及备集群;The cluster obtaining module 704 is configured to obtain a corresponding primary cluster and a standby cluster according to the target data identifier;
运行信息获取模块706,用于获取主集群的运行信息,运行信息包括运行正常或运行异常;The operation information obtaining module 706 is configured to obtain running information of the primary cluster, where the running information includes normal operation or abnormal operation;
第一响应请求发送模块708,若主集群运行异常,根据数据请求向备集群发送第一响应请求,第一响应请求携带有目标数据标识;The first response request sending module 708, if the primary cluster runs abnormally, sends a first response request to the standby cluster according to the data request, where the first response request carries the target data identifier;
响应信息接收模块710,用于接收备集群根据第一响应请求发送的第一数据响应信息并将第一数据响应信息返回给客户端。The response information receiving module 710 is configured to receive the first data response information sent by the standby cluster according to the first response request, and return the first data response information to the client.
在一些实施例中,如图8所示,集群得到模块704包括:In some embodiments, as shown in FIG. 8, the cluster obtaining module 704 includes:
计算单元802,用于根据目标数据标识进行哈希运算,得到目标数据标识对应的哈希值;The calculating unit 802 is configured to perform a hash operation according to the target data identifier, and obtain a hash value corresponding to the target data identifier.
取余单元804,用于对哈希值以及预设的数值进行取余运算,得到目标键值;a remainder unit 804, configured to perform a remainder operation on the hash value and the preset value to obtain a target key value;
集群得到单元806,用于根据目标键值获取对应的主集群以及备集群。The cluster obtaining unit 806 is configured to obtain a corresponding primary cluster and a standby cluster according to the target key value.
在一些实施例中,如图9所示,集群得到单元806包括:In some embodiments, as shown in FIG. 9, the cluster obtaining unit 806 includes:
目标标识获取子单元902,用于根据键值与集群标识的对应关系获取目标键值对应的目标集群标识;The target identifier obtaining sub-unit 902 is configured to obtain a target cluster identifier corresponding to the target key value according to the correspondence between the key value and the cluster identifier.
集群获取子单元904,用于根据集群标识与集群的对应关系获取目标集群标识对应的主集群以及备集群。The cluster obtaining sub-unit 904 is configured to obtain the primary cluster and the standby cluster corresponding to the target cluster identifier according to the correspondence between the cluster identifier and the cluster.
在一些实施例中,如图10所示,数据请求处理装置还包括:In some embodiments, as shown in FIG. 10, the data request processing apparatus further includes:
权重获取模块1002,用于获取集群标识对应的权重。The weight obtaining module 1002 is configured to obtain a weight corresponding to the cluster identifier.
对应关系获取模块1004,用于根据权重以及预设的数值为集群标识分配对应的键值,得到键值与集群标识的对应关系。The correspondence relationship obtaining module 1004 is configured to allocate a corresponding key value to the cluster identifier according to the weight and the preset value, and obtain a correspondence between the key value and the cluster identifier.
在一些实施例中,如图11所示,运行信息获取模块706包括:In some embodiments, as shown in FIG. 11, the operation information obtaining module 706 includes:
第二响应请求发送单元1102,用于根据数据请求向主集群发送第二响应请求,第二响应请求携带有目标数据标识。The second response request sending unit 1102 is configured to send a second response request to the primary cluster according to the data request, where the second response request carries the target data identifier.
运行信息获取单元1104,用于当接收到主集群根据第二响应请求返回的第二响应信息时,运行信息为运行正常,当未接收到主集群根据第二响应请求返回的第二响应信息时,运行信息为运行异常。The operation information obtaining unit 1104 is configured to: when receiving the second response information returned by the primary cluster according to the second response request, the running information is normal, when the second response information returned by the primary cluster according to the second response request is not received. The running information is abnormal.
关于数据请求处理装置的具体限定可以参见上文中对于数据请求处理方法的限定,在此 不再赘述。上述数据请求处理装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中,也可以以软件形式存储于计算机设备中的存储器中,以便于处理器调用执行以上各个模块对应的操作。For specific definitions of the data request processing apparatus, reference may be made to the above definition of the data request processing method, and details are not described herein again. The various modules in the above data request processing device may be implemented in whole or in part by software, hardware, and combinations thereof. Each of the above modules may be embedded in or independent of the processor in the computer device, or may be stored in a memory in the computer device in a software form, so that the processor invokes the operations corresponding to the above modules.
如图12所示,为一个实施例中计算机设备的内部结构图,该计算机设备可以是服务器,该计算机设备通过系统连接总线连接处理器、非易失性存储介质、内存储器和网络接口。非易失性存储介质可以是非易失性计算机可读存储介质。该计算机设备的非易失性存储介质可存储操作系统和计算机可读指令,该计算机可读指令被执行时,可使得处理器执行一种数据请求处理方法。该计算机设备的处理器用于提供计算和控制能力,支撑整个计算机设备的运行。该内存储器中可储存有计算机可读指令,该计算机可读指令被处理器执行时,可使得处理器执行一种数据请求处理方法。计算机设备的网络接口用于进行网络通信,如接收数据请求,发送数据响应请求等。As shown in FIG. 12, which is an internal structural diagram of a computer device in an embodiment, the computer device may be a server connected to a processor, a non-volatile storage medium, an internal memory, and a network interface through a system connection bus. The non-volatile storage medium can be a non-transitory computer readable storage medium. The non-volatile storage medium of the computer device can store an operating system and computer readable instructions that, when executed, cause the processor to perform a data request processing method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The internal memory can store computer readable instructions that, when executed by the processor, cause the processor to perform a data request processing method. The network interface of the computer device is used for network communication, such as receiving a data request, sending a data response request, and the like.
本领域技术人员可以理解,图12中示出的结构,仅仅是与本申请方案相关的部分结构的框图,并不构成对本申请方案所应用于其上的计算机设备的限定,具体的计算机设备可以包括比图中所示更多或更少的部件,或者组合某些部件,或者具有不同的部件布置。It will be understood by those skilled in the art that the structure shown in FIG. 12 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation of the computer device to which the solution of the present application is applied. The specific computer device may It includes more or fewer components than those shown in the figures, or some components are combined, or have different component arrangements.
在一些实施例中,本申请提供的数据请求处理装置可以实现为一种计算机可读指令的形式,计算机可读指令可在如图12所示的计算机设备上运行,所述计算机设备的非易失性存储介质可存储组成该数据请求处理装置的各个程序模块,比如图7中的数据请求接收模块702、集群得到模块704、运行信息获取模块706、第一响应请求发送模块708以及响应信息接收模块710。各个程序模块中包括计算机可读指令,所述计算机可读指令用于使所述计算机设备执行本说明书中描述的本申请各个实施例的数据请求处理方法中的步骤,例如,所述计算机设备可以通过如图7所示的数据请求处理装置中的数据请求接收模块702接收客户端发送的数据请求,数据请求携带有目标数据标识,通过集群得到模块704根据目标数据标识得到对应的主集群以及备集群,通过运行信息获取模块706获取主集群的运行信息,运行信息包括运行正常或运行异常,通过第一响应请求发送模块708根据数据请求向备集群发送第一响应请求,第一响应请求携带有目标数据标识,通过响应信息接收模块710接收备集群根据第一响应请求发送的第一数据响应信息并将第一数据响应信息返回给客户端。In some embodiments, the data request processing apparatus provided herein can be implemented in the form of a computer readable instruction that can be run on a computer device as shown in FIG. The dysfunctional storage medium can store various program modules constituting the data request processing device, such as the data request receiving module 702, the cluster obtaining module 704, the running information acquiring module 706, the first response request sending module 708, and the response information receiving device in FIG. Module 710. Each of the program modules includes computer readable instructions for causing the computer device to perform the steps in the data request processing method of various embodiments of the present application described in this specification, for example, the computer device can The data request receiving module 702 in the data request processing device shown in FIG. 7 receives the data request sent by the client, and the data request carries the target data identifier, and the cluster obtaining module 704 obtains the corresponding primary cluster and the backup according to the target data identifier. The cluster obtains the running information of the primary cluster by using the running information obtaining module 706. The running information includes a normal operation or an abnormal operation. The first response request sending module 708 sends a first response request to the standby cluster according to the data request, where the first response request carries The target data identifier is received by the response information receiving module 710, and the first data response information sent by the standby cluster according to the first response request is returned to the client.
一种计算机设备,包括存储器和一个或多个处理器,存储器中储存有计算机可读指令,计算机可读指令被处理器执行时,使得一个或多个处理器执行以下步骤:接收客户端发送的数据请求,数据请求携带有目标数据标识;根据目标数据标识得到对应的主集群以及备集群;获取主集群的运行信息,运行信息包括运行正常或运行异常;若主集群运行异常,根据数据 请求向备集群发送第一响应请求,第一响应请求携带有目标数据标识;及,接收备集群根据第一响应请求发送的第一数据响应信息并将第一数据响应信息返回给客户端。A computer device comprising a memory and one or more processors, the memory storing computer readable instructions, the computer readable instructions being executed by the processor, causing the one or more processors to perform the steps of: receiving a Data request, the data request carries the target data identifier; the corresponding primary cluster and the standby cluster are obtained according to the target data identifier; the running information of the primary cluster is obtained, and the running information includes normal operation or abnormal operation; if the primary cluster runs abnormally, according to the data request The standby cluster sends a first response request, where the first response request carries the target data identifier; and the first data response information sent by the standby cluster according to the first response request and the first data response information is returned to the client.
一个或多个存储有计算机可读指令的非易失性计算机可读存储介质,计算机可读指令被一个或多个处理器执行时,使得一个或多个处理器执行以下步骤:接收客户端发送的数据请求,数据请求携带有目标数据标识;根据目标数据标识得到对应的主集群以及备集群;获取主集群的运行信息,运行信息包括运行正常或运行异常;若主集群运行异常,根据数据请求向备集群发送第一响应请求,第一响应请求携带有目标数据标识;及,接收备集群根据第一响应请求发送的第一数据响应信息并将第一数据响应信息返回给客户端。One or more non-transitory computer readable storage mediums storing computer readable instructions, when executed by one or more processors, cause one or more processors to perform the steps of: receiving a client to send The data request carries the target data identifier; the corresponding primary cluster and the standby cluster are obtained according to the target data identifier; the running information of the primary cluster is obtained, and the running information includes normal operation or abnormal operation; if the primary cluster runs abnormally, according to the data request Sending a first response request to the standby cluster, where the first response request carries the target data identifier; and receiving the first data response information sent by the standby cluster according to the first response request, and returning the first data response information to the client.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程,是可以通过计算机可读指令来指令相关的硬件来完成,所述的计算机可读指令可存储于一非易失性计算机可读取存储介质中,该计算机可读指令在执行时,可包括如上述各方法的实施例的流程。其中,本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用,均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限,RAM以多种形式可得,诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。One of ordinary skill in the art can understand that all or part of the process of implementing the above embodiments can be completed by computer readable instructions, which can be stored in a non-volatile computer. The readable storage medium, which when executed, may include the flow of an embodiment of the methods as described above. Any reference to a memory, storage, database or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. Non-volatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of formats, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization chain. Synchlink DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).
以上实施例的各技术特征可以进行任意的组合,为使描述简洁,未对上述实施例中的各个技术特征所有可能的组合都进行描述,然而,只要这些技术特征的组合不存在矛盾,都应当认为是本说明书记载的范围。The technical features of the above embodiments may be arbitrarily combined. For the sake of brevity of description, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, It is considered to be the range described in this specification.
以上实施例仅表达了本申请的几种实施方式,其描述较为具体和详细,但并不能因此而理解为对本申请专利范围的限制。应当指出的是,对于本领域的普通技术人员来说,在不脱离本申请构思的前提下,还可以做出若干变形和改进,这些都属于本申请的保护范围。因此,本申请专利的保护范围应以所附权利要求为准。The above embodiments are merely illustrative of several embodiments of the present application, and the description thereof is more specific and detailed, but is not to be construed as limiting the scope of the claims. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the present application. Therefore, the scope of the invention should be determined by the appended claims.

Claims (20)

  1. 一种数据请求处理方法,包括:A data request processing method includes:
    接收客户端发送的数据请求,所述数据请求携带有目标数据标识;Receiving a data request sent by the client, where the data request carries a target data identifier;
    根据所述目标数据标识得到对应的主集群以及备集群;Obtaining a corresponding primary cluster and a standby cluster according to the target data identifier;
    获取所述主集群的运行信息,所述运行信息包括运行正常或运行异常;Obtaining operation information of the primary cluster, where the running information includes a normal operation or an abnormal operation;
    若所述主集群运行异常,根据所述数据请求向所述备集群发送第一响应请求,所述第一响应请求携带有所述目标数据标识;及And if the primary cluster runs abnormally, sending a first response request to the standby cluster according to the data request, where the first response request carries the target data identifier;
    接收所述备集群根据所述第一响应请求发送的第一数据响应信息并将所述第一数据响应信息返回给所述客户端。Receiving, by the standby cluster, first data response information sent according to the first response request, and returning the first data response information to the client.
  2. 根据权利要求1所述的方法,其特征在于,所述根据目标数据标识得到对应的主集群以及备集群包括:The method according to claim 1, wherein the obtaining the corresponding primary cluster and the standby cluster according to the target data identifier comprises:
    根据所述目标数据标识进行哈希运算,得到所述目标数据标识对应的哈希值;Performing a hash operation according to the target data identifier to obtain a hash value corresponding to the target data identifier;
    对所述哈希值以及预设的数值进行取余运算,得到目标键值;Performing a remainder operation on the hash value and the preset value to obtain a target key value;
    根据所述目标键值获取对应的主集群以及备集群。Obtaining a corresponding primary cluster and a standby cluster according to the target key value.
  3. 根据权利要求2所述的方法,其特征在于,所述根据所述目标键值获取对应的主集群以及备集群包括:The method according to claim 2, wherein the obtaining the corresponding primary cluster and the standby cluster according to the target key value comprises:
    根据键值与集群标识的对应关系获取所述目标键值对应的目标集群标识;及Obtaining, according to a correspondence between the key value and the cluster identifier, a target cluster identifier corresponding to the target key value; and
    根据集群标识与集群的对应关系获取所述目标集群标识对应的主集群以及备集群。The primary cluster and the standby cluster corresponding to the target cluster identifier are obtained according to the correspondence between the cluster identifier and the cluster.
  4. 根据权利要求3所述的方法,其特征在于,还包括:The method of claim 3, further comprising:
    获取集群标识对应的权重;及Obtain the weight corresponding to the cluster identifier; and
    根据所述权重以及所述预设的数值为所述集群标识分配对应的键值,得到所述键值与集群标识的对应关系。And assigning a corresponding key value to the cluster identifier according to the weight and the preset value, and obtaining a correspondence between the key value and the cluster identifier.
  5. 根据权利要求1~4任一项所述的方法,其特征在于,所述获取所述主集群的运行信息包括:The method according to any one of claims 1 to 4, wherein the obtaining the operation information of the primary cluster comprises:
    根据所述数据请求向所述主集群发送第二响应请求,所述第二响应请求携带有所述目标数据标识;及Sending a second response request to the primary cluster according to the data request, where the second response request carries the target data identifier; and
    当接收到所述主集群根据所述第二响应请求返回的第二响应信息时,所述运行信息为运行正常,当未接收到所述主集群根据所述第二响应请求返回的第二响应信息时,所述运行信息为运行异常。When the second response information returned by the primary cluster according to the second response request is received, the running information is normal, when the second response returned by the primary cluster according to the second response request is not received. The information is abnormal when the information is running.
  6. 一种数据请求处理装置,包括:A data request processing apparatus includes:
    数据请求接收模块,用于接收客户端发送的数据请求,所述数据请求携带有目标数据标识;a data request receiving module, configured to receive a data request sent by the client, where the data request carries a target data identifier;
    集群得到模块,用于根据所述目标数据标识得到对应的主集群以及备集群;The cluster obtaining module is configured to obtain a corresponding primary cluster and a standby cluster according to the target data identifier;
    运行信息获取模块,用于获取所述主集群的运行信息,所述运行信息包括运行正常或运行异常;The operation information obtaining module is configured to obtain running information of the primary cluster, where the running information includes normal operation or abnormal operation;
    第一响应请求发送模块,若所述主集群运行异常,根据所述数据请求向所述备集群发送第一响应请求,所述第一响应请求携带有所述目标数据标识;及a first response request sending module, if the primary cluster runs abnormally, sending a first response request to the standby cluster according to the data request, where the first response request carries the target data identifier;
    响应信息接收模块,用于接收所述备集群根据所述第一响应请求发送的第一数据响应信息并将所述第一数据响应信息返回给所述客户端。The response information receiving module is configured to receive first data response information sent by the standby cluster according to the first response request, and return the first data response information to the client.
  7. 根据权利要求6所述的装置,其特征在于,所述集群得到模块包括:The apparatus according to claim 6, wherein the cluster obtaining module comprises:
    计算单元,用于根据所述目标数据标识进行哈希运算,得到所述目标数据标识对应的哈希值;a calculating unit, configured to perform a hash operation according to the target data identifier, to obtain a hash value corresponding to the target data identifier;
    取余单元,用于对所述哈希值以及预设的数值进行取余运算,得到目标键值;及a suffix unit for performing a remainder operation on the hash value and the preset value to obtain a target key value;
    集群得到单元,用于根据所述目标键值获取对应的主集群以及备集群。The cluster obtaining unit is configured to obtain a corresponding primary cluster and a standby cluster according to the target key value.
  8. 根据权利要求7所述的装置,其特征在于,所述集群得到单元包括:The apparatus according to claim 7, wherein the cluster obtaining unit comprises:
    目标标识获取子单元,用于根据键值与集群标识的对应关系获取所述目标键值对应的目标集群标识;a target identifier obtaining subunit, configured to acquire, according to a correspondence between the key value and the cluster identifier, a target cluster identifier corresponding to the target key value;
    集群获取子单元,用于根据集群标识与集群的对应关系获取所述目标集群标识对应的主集群以及备集群。The cluster obtaining sub-unit is configured to obtain the primary cluster and the standby cluster corresponding to the target cluster identifier according to the correspondence between the cluster identifier and the cluster.
  9. 根据权利要求8所述的装置,其特征在于,还包括:The device according to claim 8, further comprising:
    权重获取模块,用于获取集群标识对应的权重;及a weight obtaining module, configured to obtain a weight corresponding to the cluster identifier; and
    对应关系获取模块,用于根据所述权重以及所述预设的数值为所述集群标识分配对应的键值,得到所述键值与集群标识的对应关系。The corresponding relationship obtaining module is configured to allocate a corresponding key value to the cluster identifier according to the weight and the preset value, and obtain a correspondence between the key value and the cluster identifier.
  10. 根据权利要求6~9任一项所述的装置,其特征在于,所述运行信息获取模块包括:The device according to any one of claims 6 to 9, wherein the operation information acquisition module comprises:
    第二响应请求发送单元,用于根据所述数据请求向所述主集群发送第二响应请求,所述第二响应请求携带有所述目标数据标识;及a second response request sending unit, configured to send a second response request to the primary cluster according to the data request, where the second response request carries the target data identifier;
    运行信息获取单元,用于当接收到所述主集群根据所述第二响应请求返回的第二响应信息时,所述运行信息为运行正常,当未接收到所述主集群根据所述第二响应请求返回的第二响应信息时,所述运行信息为运行异常。a running information obtaining unit, configured to: when the second response information returned by the primary cluster according to the second response request is received, the running information is normal, and when the primary cluster is not received according to the second The operation information is a running exception when responding to the second response information returned by the request.
  11. 一种计算机设备,包括存储器及一个或多个处理器,所述存储器中储存有计算机可 读指令,所述计算机可读指令被所述一个或多个处理器执行时,使得所述一个或多个处理器执行以下步骤:A computer device comprising a memory and one or more processors having stored therein computer readable instructions, the computer readable instructions being executed by the one or more processors to cause the one or more The processors perform the following steps:
    接收客户端发送的数据请求,所述数据请求携带有目标数据标识;Receiving a data request sent by the client, where the data request carries a target data identifier;
    根据所述目标数据标识得到对应的主集群以及备集群;Obtaining a corresponding primary cluster and a standby cluster according to the target data identifier;
    获取所述主集群的运行信息,所述运行信息包括运行正常或运行异常;Obtaining operation information of the primary cluster, where the running information includes a normal operation or an abnormal operation;
    若所述主集群运行异常,根据所述数据请求向所述备集群发送第一响应请求,所述第一响应请求携带有所述目标数据标识;及And if the primary cluster runs abnormally, sending a first response request to the standby cluster according to the data request, where the first response request carries the target data identifier;
    接收所述备集群根据所述第一响应请求发送的第一数据响应信息并将所述第一数据响应信息返回给所述客户端。Receiving, by the standby cluster, first data response information sent according to the first response request, and returning the first data response information to the client.
  12. 根据权利要求11所述的计算机设备,其特征在于,所述处理器所执行的所述根据目标数据标识得到对应的主集群以及备集群包括:The computer device according to claim 11, wherein the master cluster and the standby cluster obtained by the processor according to the target data identifier comprise:
    根据所述目标数据标识进行哈希运算,得到所述目标数据标识对应的哈希值;Performing a hash operation according to the target data identifier to obtain a hash value corresponding to the target data identifier;
    对所述哈希值以及预设的数值进行取余运算,得到目标键值;Performing a remainder operation on the hash value and the preset value to obtain a target key value;
    根据所述目标键值获取对应的主集群以及备集群。Obtaining a corresponding primary cluster and a standby cluster according to the target key value.
  13. 根据权利要求12所述的计算机设备,其特征在于,所述处理器所执行的所述根据所述目标键值获取对应的主集群以及备集群包括:The computer device according to claim 12, wherein the obtaining, by the processor, the corresponding primary cluster and the standby cluster according to the target key value comprises:
    根据键值与集群标识的对应关系获取所述目标键值对应的目标集群标识;及Obtaining, according to a correspondence between the key value and the cluster identifier, a target cluster identifier corresponding to the target key value; and
    根据集群标识与集群的对应关系获取所述目标集群标识对应的主集群以及备集群。The primary cluster and the standby cluster corresponding to the target cluster identifier are obtained according to the correspondence between the cluster identifier and the cluster.
  14. 根据权利要求13所述的计算机设备,其特征在于,所述处理器执行所述计算机可读指令时还执行以下步骤:The computer apparatus according to claim 13, wherein said processor further performs the following steps when said computer readable instructions are executed:
    获取集群标识对应的权重;及Obtain the weight corresponding to the cluster identifier; and
    根据所述权重以及所述预设的数值为所述集群标识分配对应的键值,得到所述键值与集群标识的对应关系。And assigning a corresponding key value to the cluster identifier according to the weight and the preset value, and obtaining a correspondence between the key value and the cluster identifier.
  15. 根据权利要求11~14任一项所述的计算机设备,其特征在于,所述处理器所执行的所述获取所述主集群的运行信息包括:The computer device according to any one of claims 11 to 14, wherein the obtaining the operation information of the primary cluster performed by the processor comprises:
    根据所述数据请求向所述主集群发送第二响应请求,所述第二响应请求携带有所述目标数据标识;及Sending a second response request to the primary cluster according to the data request, where the second response request carries the target data identifier; and
    当接收到所述主集群根据所述第二响应请求返回的第二响应信息时,所述运行信息为运行正常,当未接收到所述主集群根据所述第二响应请求返回的第二响应信息时,所述运行信息为运行异常。When the second response information returned by the primary cluster according to the second response request is received, the running information is normal, when the second response returned by the primary cluster according to the second response request is not received. The information is abnormal when the information is running.
  16. 一个或多个存储有计算机可读指令的非易失性计算机可读存储介质,所述计算机可读指令被一个或多个处理器执行时,使得所述一个或多个处理器执行以下步骤:One or more non-transitory computer readable storage mediums storing computer readable instructions, when executed by one or more processors, cause the one or more processors to perform the following steps:
    接收客户端发送的数据请求,所述数据请求携带有目标数据标识;Receiving a data request sent by the client, where the data request carries a target data identifier;
    根据所述目标数据标识得到对应的主集群以及备集群;Obtaining a corresponding primary cluster and a standby cluster according to the target data identifier;
    获取所述主集群的运行信息,所述运行信息包括运行正常或运行异常;Obtaining operation information of the primary cluster, where the running information includes a normal operation or an abnormal operation;
    若所述主集群运行异常,根据所述数据请求向所述备集群发送第一响应请求,所述第一响应请求携带有所述目标数据标识;及And if the primary cluster runs abnormally, sending a first response request to the standby cluster according to the data request, where the first response request carries the target data identifier;
    接收所述备集群根据所述第一响应请求发送的第一数据响应信息并将所述第一数据响应信息返回给所述客户端。Receiving, by the standby cluster, first data response information sent according to the first response request, and returning the first data response information to the client.
  17. 根据权利要求16所述的存储介质,其特征在于,所述处理器所执行的所述根据目标数据标识得到对应的主集群以及备集群包括:The storage medium according to claim 16, wherein the master cluster and the standby cluster obtained by the processor according to the target data identifier comprise:
    根据所述目标数据标识进行哈希运算,得到所述目标数据标识对应的哈希值;Performing a hash operation according to the target data identifier to obtain a hash value corresponding to the target data identifier;
    对所述哈希值以及预设的数值进行取余运算,得到目标键值;Performing a remainder operation on the hash value and the preset value to obtain a target key value;
    根据所述目标键值获取对应的主集群以及备集群。Obtaining a corresponding primary cluster and a standby cluster according to the target key value.
  18. 根据权利要求17所述的存储介质,其特征在于,所述处理器所执行的所述根据所述目标键值获取对应的主集群以及备集群包括:The storage medium according to claim 17, wherein the master cluster and the standby cluster obtained by the processor according to the target key value include:
    根据键值与集群标识的对应关系获取所述目标键值对应的目标集群标识;及Obtaining, according to a correspondence between the key value and the cluster identifier, a target cluster identifier corresponding to the target key value; and
    根据集群标识与集群的对应关系获取所述目标集群标识对应的主集群以及备集群。The primary cluster and the standby cluster corresponding to the target cluster identifier are obtained according to the correspondence between the cluster identifier and the cluster.
  19. 根据权利要求18所述的存储介质,其特征在于,所述处理器执行所述计算机可读指令时还执行以下步骤:The storage medium of claim 18, wherein the processor further performs the following steps when the computer readable instructions are executed:
    获取集群标识对应的权重;及Obtain the weight corresponding to the cluster identifier; and
    根据所述权重以及所述预设的数值为所述集群标识分配对应的键值,得到所述键值与集群标识的对应关系。And assigning a corresponding key value to the cluster identifier according to the weight and the preset value, and obtaining a correspondence between the key value and the cluster identifier.
  20. 根据权利要求16~19任一项所述的存储介质,其特征在于,所述处理器所执行的所述获取所述主集群的运行信息包括:The storage medium according to any one of claims 16 to 19, wherein the acquiring the operation information of the primary cluster performed by the processor comprises:
    根据所述数据请求向所述主集群发送第二响应请求,所述第二响应请求携带有所述目标数据标识;及Sending a second response request to the primary cluster according to the data request, where the second response request carries the target data identifier; and
    当接收到所述主集群根据所述第二响应请求返回的第二响应信息时,所述运行信息为运行正常,当未接收到所述主集群根据所述第二响应请求返回的第二响应信息时,所述运行信息为运行异常。When the second response information returned by the primary cluster according to the second response request is received, the running information is normal, when the second response returned by the primary cluster according to the second response request is not received. The information is abnormal when the information is running.
PCT/CN2018/077132 2017-11-14 2018-02-24 Method and device for processing data request, computer apparatus, and storage medium WO2019095577A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711122222.0 2017-11-14
CN201711122222.0A CN108011929B (en) 2017-11-14 2017-11-14 Data request processing method and device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
WO2019095577A1 true WO2019095577A1 (en) 2019-05-23

Family

ID=62051503

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/077132 WO2019095577A1 (en) 2017-11-14 2018-02-24 Method and device for processing data request, computer apparatus, and storage medium

Country Status (2)

Country Link
CN (1) CN108011929B (en)
WO (1) WO2019095577A1 (en)

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109492013B (en) * 2018-11-02 2021-01-26 京东数字科技控股有限公司 Data processing method, device and system applied to database cluster
CN109462646B (en) * 2018-11-12 2021-11-19 平安科技(深圳)有限公司 Abnormal response method and equipment
CN113039751B (en) * 2018-11-27 2023-10-31 蚂蚁矿池科技有限公司 Fault notification method, device and equipment
CN109302498B (en) * 2018-11-29 2021-03-23 杭州迪普科技股份有限公司 Network resource access method and device
CN109769028A (en) * 2019-01-25 2019-05-17 深圳前海微众银行股份有限公司 Redis cluster management method, device, equipment and readable storage medium storing program for executing
CN109981459B (en) * 2019-02-28 2021-02-19 联想(北京)有限公司 Information sending method, client and computer readable storage medium
CN110012111B (en) * 2019-04-16 2022-04-15 昆山丘钛微电子科技股份有限公司 Data service cluster system and data processing method
CN112445859A (en) * 2019-08-30 2021-03-05 北京数聚鑫云信息技术有限公司 Data management method, device and system
CN110636122A (en) * 2019-09-11 2019-12-31 中移(杭州)信息技术有限公司 Distributed storage method, server, system, electronic device, and storage medium
CN112565327B (en) * 2019-09-26 2022-09-30 广州虎牙科技有限公司 Access flow forwarding method, cluster management method and related device
CN110765109A (en) * 2019-10-24 2020-02-07 中国建设银行股份有限公司 Service request response method, device, equipment and storage medium
CN113051110A (en) * 2019-12-27 2021-06-29 中国移动通信集团湖南有限公司 Cluster switching method, device and equipment
JP7429609B2 (en) * 2020-06-04 2024-02-08 株式会社東海理化電機製作所 Processing equipment, systems, programs, and collation equipment
CN111901395B (en) * 2020-07-07 2024-03-22 青岛海尔科技有限公司 Multi-cluster switching method and device
CN112436974B (en) * 2020-07-29 2021-12-24 上海哔哩哔哩科技有限公司 CDN data resource consistency detection method and device and computer equipment
CN112199401B (en) * 2020-11-30 2021-07-23 阿里云计算有限公司 Data request processing method, device, server, system and storage medium
CN112565025A (en) * 2021-02-19 2021-03-26 北京声智科技有限公司 Data transmission method and device
CN113114481A (en) * 2021-02-22 2021-07-13 广东电网有限责任公司广州供电局 Data operation method and device, computer equipment and storage medium
CN114422567A (en) * 2021-12-09 2022-04-29 阿里巴巴(中国)有限公司 Data request processing method, device, system, computer equipment and medium
CN114785863A (en) * 2022-04-21 2022-07-22 中国工商银行股份有限公司 Service request processing method, device, equipment and storage medium based on Eureka
CN115827757B (en) * 2022-11-30 2024-03-12 西部科学城智能网联汽车创新中心(重庆)有限公司 Data operation method and device for multi-HBase cluster
CN116016257B (en) * 2023-01-28 2023-06-20 鹏城实验室 State monitoring method, device, equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103312541A (en) * 2013-05-28 2013-09-18 浪潮电子信息产业股份有限公司 Management method of high-availability mutual backup cluster
CN103607312A (en) * 2013-11-29 2014-02-26 广州华多网络科技有限公司 Data request processing method and system for server system
US20140188833A1 (en) * 2012-12-28 2014-07-03 Fujitsu Limited Information processing device, information processing terminal, recording medium storing information search program, and information search method
CN104219085A (en) * 2014-08-07 2014-12-17 深圳市同洲电子股份有限公司 Proxy server and data processing method and system of database
CN104424283A (en) * 2013-08-30 2015-03-18 阿里巴巴集团控股有限公司 Data migration system and data migration method

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9449038B2 (en) * 2012-11-26 2016-09-20 Amazon Technologies, Inc. Streaming restore of a database from a backup system
CN104202375B (en) * 2014-08-22 2017-10-24 广州华多网络科技有限公司 The method and system of synchrodata
CN104580432A (en) * 2014-12-23 2015-04-29 上海帝联信息科技股份有限公司 Memcached system, memory cache data provision method and device, memory cache data maintenance method and device as well as cluster maintenance method and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140188833A1 (en) * 2012-12-28 2014-07-03 Fujitsu Limited Information processing device, information processing terminal, recording medium storing information search program, and information search method
CN103312541A (en) * 2013-05-28 2013-09-18 浪潮电子信息产业股份有限公司 Management method of high-availability mutual backup cluster
CN104424283A (en) * 2013-08-30 2015-03-18 阿里巴巴集团控股有限公司 Data migration system and data migration method
CN103607312A (en) * 2013-11-29 2014-02-26 广州华多网络科技有限公司 Data request processing method and system for server system
CN104219085A (en) * 2014-08-07 2014-12-17 深圳市同洲电子股份有限公司 Proxy server and data processing method and system of database

Also Published As

Publication number Publication date
CN108011929B (en) 2020-08-25
CN108011929A (en) 2018-05-08

Similar Documents

Publication Publication Date Title
WO2019095577A1 (en) Method and device for processing data request, computer apparatus, and storage medium
US10298457B2 (en) Synchronization of configuration file of virtual application distribution chassis
WO2019075978A1 (en) Data transmission method and apparatus, computer device, and storage medium
WO2021051880A1 (en) Resource data acquisition method and apparatus, computer device and storage medium
AU2014235853B2 (en) Local server for synced online content management system
JP2019204524A (en) Application service delivery through application service avatar
WO2017166513A1 (en) Container technique-based method and system for application service seamless migration
AU2015311887B9 (en) Distributed data synchronization and conflict resolution
AU2015370268B2 (en) System and method for discovering a lan synchronization candidate for a synchronized content management system
WO2019075773A1 (en) Data processing method and apparatus, computer device and storage medium
TWI716822B (en) Method and device for correcting transaction causality, and electronic equipment
WO2022222438A1 (en) Profile state synchronization method and apparatus, terminal, server, and storage medium
CN112351068A (en) Information synchronization method, system, device, computer equipment and storage medium
US11848994B2 (en) Communication protocols for an online content management system
JP2015513721A (en) Retrieving availability information from public calendars
US10545667B1 (en) Dynamic data partitioning for stateless request routing
US20240015135A1 (en) Domain management and synchronization system
EP3349416B1 (en) Relationship chain processing method and system, and storage medium
CN114827171A (en) Information synchronization method and device, computer equipment and storage medium
CN114116889A (en) Method and device for synchronizing material data, computer equipment and storage medium
CN114500485A (en) Data processing method and device
CN117294752A (en) Configuration file synchronization method and device

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 18/08/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 18877706

Country of ref document: EP

Kind code of ref document: A1