WO2020259086A1

WO2020259086A1 - Distributed architecture

Info

Publication number: WO2020259086A1
Application number: PCT/CN2020/088833
Authority: WO
Inventors: 卢道和; 杨军; 陈广胜; 胡盼盼; 杨俊杰; 饶俊明
Original assignee: 深圳前海微众银行股份有限公司
Priority date: 2019-06-25
Filing date: 2020-05-06
Publication date: 2020-12-30
Also published as: CN110225138B; CN110225138A

Abstract

The present invention relates to the field of financial technology. Disclosed is a distributed architecture, the distributed architecture comprising: N data centers and M groups of data center nodes. Each data center comprises X data center nodes, and each group of data center nodes comprises Y data center nodes, wherein Y data center nodes in each group of data center nodes are located in Y different data centers, and each data center node comprises a database server for storing client data of Z clients and/or an application server for storing an application system for processing all services of the Z clients. A data center node comprised in each data center of the distributed architecture takes a client as a dimension, and the data center nodes are divided into groups, wherein each data center node has an independent application system required by service processing, so that high availability can still be maintained when the data center nodes in the distributed architecture break down, processing pressure and fault risks of a single data center are dispersed, and the influence range of faults is effectively reduced.

Description

A distributed architecture

Cross references to related applications

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on June 25, 2019, with an application number of 201910553289. 2, and the application name is "a distributed architecture", the entire content of which is incorporated into this application by reference.

Technical field

The embodiment of the present invention relates to the field of financial technology (Fintech), in particular to a distributed architecture.

Background technique

With the development of computer technology, more and more technologies are applied in the financial field. The traditional financial industry is gradually transforming to Fintech. Message storage technology is no exception. However, due to the security and real-time nature of the financial and payment industries Requirements, but also higher requirements for technology.

At present, the common distributed architecture mainly adopts the centralized loosely coupled architecture. For this centralized loosely coupled architecture, because the customer's business is highly interdependent, according to the barrel principle, the availability and performance of the entire architecture depends on the shortest node, so , The performance of each node can only solve the processing performance of its own application and cannot realize the sharing of system load and resources between nodes. Secondly, the current network technology cannot guarantee the absolute stability and availability of long-distance communication quality. When the network fails or data synchronization is abnormal due to other reasons, the centralized loosely coupled architecture may cause customer data to be inaccessible, and there is a risk of failure.

Summary of the invention

The embodiment of the present invention provides a distributed architecture to ensure the overall availability and scalability of the architecture, and reduce the scope of the failure risk.

In the first aspect, a distributed architecture provided by an embodiment of the present invention includes: N data centers and M groups of data center nodes;

Each data center includes X data center nodes, each group of data center nodes includes Y data center nodes, and Y data center nodes in each group of data center nodes are located in Y different data centers; each data center node includes A database server storing customer data of Z customers and/or an application server storing an application system that processes all services of the Z customers;

Wherein, the database server storing the customer data of each customer and/or the application server storing the application system processing all the services of the Z customers are stored in one data center node, and at least Q pieces of data in the N data centers The centers are located in different cities, the Y data center nodes in each group of data center nodes include one master data center node and at least two slave data center nodes, and N, M, X, Y, Z, and Q are positive integers.

In the above technical solution, since the data center nodes included in each data center of the distributed architecture are divided into groups based on the customer dimension, each data center node has an independent business processing application system and/or stores customers Data can maintain high availability even when a data center node fails in a distributed architecture, disperse the processing pressure and failure risk of a single data center, effectively reduce the scope of the failure, and all Y different data centers store customer data And/or application system, it also realizes that customer data is more active and/or application is more active, even if there is a failure, it can seamlessly provide services, reducing the scope of the risk of failure.

Optionally, the Y data center nodes in each group of data center nodes include one primary data center node, at least one secondary data center node in the same city as the primary data center node, and at least one secondary data center node that is connected to the primary data center node. The central node is a remote slave data center node.

Optionally, the customer data of the Z customers stored in the master data center node and the slave data center node are the same, and/or the stored application systems that process all the services of the Z customers are the same.

Optionally, the physical resources among the Y data center nodes are isolated from each other.

Optionally, the Y data center nodes preset one data center node for grayscale publishing.

Optionally, the physical resources in each data center node include multiple database servers and multiple application servers;

The application systems are grouped according to different application domains, and different groups do not share application servers and do not share database servers.

Optionally, the distributed architecture is horizontally expanded by increasing the number of data center nodes in each group of data center nodes.

Optionally, the distributed architecture is vertically expanded by increasing the computing resources of the preset data center nodes in each group of data center nodes; or by temporarily allocating the computing resources in the reserved computing resource pool to The preset data center nodes in each group of data center nodes perform vertical expansion of the distributed architecture.

Optionally, the distributed architecture further includes a global positioning system;

The global positioning system uses a preset weighted random algorithm to manage the fragmentation strategy of the customer, and locate the data center node where the customer data is stored.

Optionally, the global positioning system communicates with application systems in each data center node through a message bus.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative work, other drawings can be obtained from these drawings.

FIG. 1 is a schematic structural diagram of a distributed architecture provided by an embodiment of the present invention;

2 is a schematic structural diagram of a distributed architecture provided by an embodiment of the present invention;

3 is a schematic diagram of a data center node provided by an embodiment of the present invention;

Fig. 4 is a schematic diagram of a global positioning system provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

FIG. 1 exemplarily shows a schematic structural diagram of a distributed architecture provided by an embodiment of the present invention. As shown in FIG. 1, the distributed architecture may include N data centers (Internet Data Center, IDC) and M groups of data Data Center Node (DCN).

In the embodiment of the present invention, the data center (Internet Data Center, IDC) is a physical unit for the planning and management of a new generation of Internet architecture in the financial field. It has network throughput capabilities and security protection capabilities. The data center can be selected according to the location The modules constitute the physical architecture of the data center. The standardization of each module includes its network architecture, physical deployment, hardware equipment models, etc., but does not include its capacity; the capacity of the module can be expanded horizontally as needed.

The data center node is the logical unit of planning and management of the new generation Internet architecture in the financial field. Each data center node is a node with independent physical resources and self-contained application logic, used to carry a specific customer group or provide a set of specific services. Data center nodes have independent physical computing and storage resources. Different nodes do not share physical computing and storage resources. Different nodes share data center-level resources, mainly including: data center infrastructure, basic network, and data center level Public services (for example: message bus, etc.).

In the specific implementation process, data center nodes can be divided into two types of services according to different service objects: one is customer service: the external services provided by the bank to various types of bank customers; the other is the bank Back-end management services: internal services used by the bank itself, such as general ledger, management accounting, etc.

In the embodiment of the present invention, as shown in FIG. 1, each data center includes X data center nodes, each group of data center nodes includes Y data center nodes, and Y data center nodes in each group of data center nodes are located at Y Different data centers; each data center node includes a database server for storing customer data of Z customers and/or an application server for storing application systems for processing all services of Z customers. Among them, the database server that stores the customer data of each customer and/or the application server that stores the application systems that process all the services of the Z customers are stored in one data center node, and at least Q of the N data centers are located in different cities , The Y data center nodes in each group of data center nodes include one master data center node and at least two slave data center nodes, and N, M, X, Y, Z, and Q are positive integers. The customer data of Z customers stored in the main data center node and the slave data center node are the same as the application system that processes all the services of the Z customers.

In the embodiment of the present invention, the number of data centers in the same city can be selected as multiple (for example, N or less than N). When the number of data centers in the same city is multiple, in order to reduce excessive resource consumption, optional The number of Y is less than N. In this way, in the M groups of data center nodes, each group of data center nodes only needs to include Y data center nodes. It is not necessary for each group of data center nodes to contain N data center nodes. When a data center node fails, a backup data center node provides services, which can prevent excessive consumption of resources.

For better understanding, as shown in Figure 1, there are currently N data centers (IDCs), such as 5, M groups of data center nodes (DCN), and DCN group 1 can optionally include three DCNs, one of which is located in IDC1 Among them, one is located in IDC2, and the other is located in IDC3. The DCNs in IDC1 and IDC2 each contain a database server that stores customer data and an application server that stores application systems that process all customer business. IDC3 only contains storage The database server of customer data, in this way, the number of data center nodes in each group of data center nodes is less than the number of data centers, which can ensure that there is a backup data center node when a data center node fails Providing services can also prevent excessive consumption of resources. This is only an implementation that can be implemented. In specific implementation, IDC3 can also deploy an application server.

In the embodiment of the present invention, it is important to emphasize that a data center node includes a database server and/or an application server, and the database server and the application server are independent of each other on two different levels and do not respond to each other. In addition, by deploying application systems on different DCNs in the same DCN group, multi-instance and active deployment of application systems can be realized. Even if the application system in a certain DCN fails, it can be provided by other DCN application systems. Services can ensure business continuity in various disaster recovery scenarios. In the existing distributed architecture, when a certain application system fails, the backup application system can only be manually started manually, which requires a long waiting time, and the obvious failure has a large impact range.

In the embodiment of the present invention, each data center node has an independent application system for processing all customer services. When the current data center node has a problem and cannot handle the customer’s business, other data center nodes in the same group can be used The application system takes over the processing, and there is no need to set up the main and standby application systems in a data center node, saving the resources of the data center node.

Further, at least Q data centers of N data centers are located in different cities. For example, in the distributed architecture as shown in FIG. 2, there are three data centers in total, of which there are two data centers in the same city and one data center for remote disaster recovery. This kind of architecture can be called a "two places and three centers" deployment architecture. Under this kind of architecture, a single data center from power supply, cooling to network connectivity is designed according to the minimum 2N level of redundancy, and requires all lines to be physically Isolation to ensure that a construction site does not have a fatal impact on the connectivity of a single data center. Therefore, when any data center in the same city fails, the architecture can support the switch between data centers in the same city.

Optionally, the Y data center nodes in each group of data center nodes include a primary data center node, at least one secondary data center node in the same city as the primary data center node, and at least one secondary data center node that is remote from the primary data center node . Deep into the data center, each data center is composed of multiple data center nodes. Each group of data center nodes is composed of a master node, a backup node in the same city, and a disaster recovery node in a remote location. This also means that each group of data center nodes also independently form a "two places and three centers" structure to ensure that the main node fails It can quickly switch to the standby node and support uninterrupted business operation to meet RTO (Recovery Time Object, Recovery Time Object) and RPO (Recovery Point Object, Recovery Point Object) related requirements. At the same time, under normal circumstances, both data centers have the master node of the data center node to survive and provide services to the outside at the same time, which can disperse the processing pressure and failure risk of a single data center, and effectively reduce the impact of failure.

Optionally, there can be three data center nodes in the same group of data center nodes. Two data center nodes can be located in the same city, and the other data center node can be located in a remote city, so that each group of data center nodes can also be formed independently. The "two locations and three centers" structure can disperse the processing pressure of a single data center and save system overhead.

For a single data center node, its physical resources include multiple database servers and multiple application servers. Application systems are grouped according to different application domains, and different groups do not share application servers and do not share database servers. The physical resources between data center nodes are isolated from each other. In other words, data center nodes can be analyzed from the composed hardware resources and deployed data and application systems. In terms of the hardware resource composition of data center nodes. As shown in Figure 3, a data center node contains hardware resources such as application servers and database servers. In the initial stage of infrastructure construction, after full evaluation of the business volume, and from the perspective of prudent management, the relevant hardware resources of a data center node are deployed in a centralized manner, fixed by 2 application server cabinets and 3 database server cabinets constitute. Two application server cabinets provide dual cabinet redundancy, ensuring that application instances are conditionally deployed to two different cabinets, avoiding the risk of single cabinet failure. For the database, each SET (data set) is composed of three SET nodes, and the three SET nodes are scattered on three database server cabinets. For the application system, the corresponding data is stored in three copies, and the three data are distributed in three different cabinets. Such a database deployment structure can fully guarantee the high availability of data. After years of practice, combined with operation and maintenance management tools such as configuration information management, banks have become more mature in the management of data center nodes, and can read and filter all resource information related to a data center node from the operation and maintenance management tools at any time and quickly. At the same time, with further business development, data center nodes are also facing certain expansion requirements. Therefore, the current data center nodes are no longer limited to centralized cabinets, but are more open and transformed into logical regional divisions. The data center becomes a huge resource pool. A data center can be integrated at any time according to actual construction needs. Any server joining or moving out of a data center node can quickly realize the elastic expansion and shrinkage of data center nodes.

In terms of customer data and application systems deployed at data center nodes. From the perspective of physical resource isolation, since the physical resources of each data center node are isolated from each other, the customer data and application systems on each data center node are naturally isolated from each other. This isolation is like a protective net, which can avoid the mutual influence between data center nodes, making each data center node like an independent BOX, which can effectively avoid the spread of influence when a problem occurs. At the same time, this isolation structure also brings convenience to the implementation of gray-scale release when the application version is updated. The application does not need to be adapted to the gray-scale release at the logic level, and can be directly released at the data center node level, which greatly reduces Design and development costs for grayscale release. From the perspective of application deployment, in order to isolate the interaction between business products as much as possible and reduce the complexity of cross-cooperation between departments, banks in practice group applications into different application domains, and different groups do not share application servers. Nor does it share the database server. In the design of application architecture, banks are currently more vertically divided according to different business products. Except for some public platform systems, for application subsystems in different application domains, their corresponding business product systems are also in most cases. They are not the same. Therefore, after resources are grouped into different application domains, the scope of impact of resource problems can be controlled within a single application domain, so as to minimize the impact of cross-business products.

Each group of data center nodes can preset a data center node for grayscale release. Through the distributed architecture with customers as the unit, the weight of customer allocation of one of the nodes can be reduced, so that this node has exactly the same application architecture, deployment architecture and resource configuration as all other nodes, but lower than the customer load of other nodes .

All application version releases and basic component upgrades are verified in grayscale on this node. Since it is fully deployed in the production environment and has the same configuration as other nodes, its gray-scale results can truly reflect the effect of the change on other nodes. At the same time, because it has a low percentage of customers (less than 10%), even if the gray-scale verification is abnormal, the relevant impact can be controlled in a small customer group.

Compared with the traditional centralized loosely coupled architecture, due to the differences in deployment methods and technical frameworks, it is more difficult to achieve gray release in a complete end-to-end environment. Because a system needs to be deeply customized to identify whether it is a gray-scale verified transaction or a normal transaction. The general approach is to make a distinction at the entrance of a system. According to the gray configuration, the customer transactions in the gray verification list are distributed to the gray verification version, and other transactions are released to the normal production version. Doing so will greatly increase the complexity and risk of implementation at the technical level. Once a mistake occurs, it may affect all normal transactions. At the same time, if the grayscale version involves changes to the database, the entire grayscale scheme will be more complicated.

Therefore, the distributed architecture with customers as the unit, through the isolation of each customer node, through standardized node deployment and the control of customer distribution weights, can easily achieve true and effective grayscale verification, thereby greatly improving the application release cycle. Reduce the dependence on the test process, through the gray production flow, directly completed the last test link of software and hardware update in the production environment.

In the embodiment of the present invention, the distributed architecture has two expansion methods, specifically:

1. Horizontally expand the distributed architecture by increasing the number of data center nodes in each group of data center nodes.

2. By increasing the computing resources of the preset data center nodes in each group of data center nodes, permanent vertical expansion of the distributed architecture; or by temporarily allocating the computing resources in the reserved computing resource pool to each group of data center nodes The preset data center node in, temporarily expands the distributed architecture vertically.

In the horizontal expansion strategy, the bank’s customer service capacity can be increased by rapidly deploying corresponding types of standard data center nodes; in the vertical expansion strategy, we have two different modes: one is permanent expansion, the other is Temporary expansion. Permanent expansion refers to vertical expansion and upgrade by increasing the computing resources of logical nodes. For example, a module was originally scheduled to serve 5 million customers. With the continuous development of business and the continuous launch of new products, while serving 5 million customers unchanged, multiple nodes generally have performance bottlenecks. Then, at this time, computing resources will be added to these nodes according to certain strategies, and the processing capabilities of the nodes will be permanently improved. Another situation is that under the new operating concepts brought about by the Internet, many temporary marketing activities such as "E-commerce 618", "Double 11", "Double 12" and so on will be encountered. For temporary resource and performance requirements, computing resources will be mounted on the corresponding node from the reserved resource pool as needed, and this node will be temporarily expanded vertically.

Further, the distributed architecture also includes a global positioning system, which uses a preset weighted random algorithm to manage fragmentation strategies for customers and locate data center nodes where customer data is stored. The global positioning system uses a weighted random algorithm to determine which data center node to store the new customer's data when creating a new customer, and in the subsequent business processing of the new customer, through a fragmented information retrieval mechanism based on customer information To locate the data center node where customer data needed for business processing is stored, that is, use the global positioning system for customer segmentation management and customer positioning.

In the embodiment of the present invention, the customer data storage nodes are allocated through the global positioning system, and the weighted random algorithm is used to implement in a distributed architecture. There is a reasonable customer distribution among data center nodes, and each data in the distributed architecture is fully utilized. In the case of the computing resources of the central node, customers are dispersed and distributed at the same time, reducing the impact on the distributed architecture when a data center node fails.

As shown in Figure 4, the global positioning system communicates with the application systems in each data center node through the message bus.

The biggest difference between the above model and the model adopted by traditional banks is the introduction of the dimension of data center nodes, which splits a data center into multiple data center nodes. In terms of architecture design, the biggest challenge brought by this difference is how to effectively solve the communication problem between systems, that is, an application subsystem will be distributed on multiple data center nodes at the same time, how does the upstream caller know which data center to visit? The downstream subsystem of the node. To solve this problem, it is impossible to make a lot of changes to all applications to adapt to this architecture. The best way is to converge and solve the problem. The message bus is a design scheme that has been proposed for a long time, and the communication between various application subsystems can be converged through the message bus. But just using the message bus is not enough. It also needs to solve the application service positioning problem of the upstream subsystem to quickly find the appropriate downstream subsystem. Therefore, the embodiment of the present invention introduces a global positioning system that provides uniform customer and service addressing functions across the bank, and returns the data center node number where the customer is located based on the customer identification information such as the customer number, card number, or account number entered. This makes it possible to obtain the data center node where the callee is located when calling between systems, and inform the upstream subsystem of which data center node's downstream subsystem to visit through the standard interface provided by the global positioning system.

In summary, based on the above-mentioned distributed architecture, the embodiment of the present invention not only realizes the overall architecture of “two places and multiple centers”, but also achieves specific results in the following three aspects: Distribute applications to multiple data center nodes through the design In the homogeneous BOX, the flexible expansion and shrinkage capabilities of a single data center node are realized; the multi-instance multi-active deployment of applications is realized, and the "two-place multi-center" solution is combined to ensure business in various disaster recovery scenarios Continuity: Through the message bus and the global positioning system, the communication problem between systems is solved, and the impact of the change of the architecture on the application is minimized.

The embodiment of the present invention shows that the distributed architecture includes: N data centers and M groups of data center nodes. Each data center includes X data center nodes, each group of data center nodes includes Y data center nodes, and Y data center nodes in each group of data center nodes are located in Y different data centers; each data center node is used for Including a database server that stores customer data of Z customers and/or an application server that stores application systems that process all services of Z customers; among them, a database server that stores customer data of each customer and/or stores and processes the Z customers The application servers of all business application systems are stored in one data center node, at least Q data centers in the N data centers are located in different cities, and the Y data center nodes in each group of data center nodes include a master For the data center node and at least two slave data center nodes, N, M, X, Y, Z, and Q are positive integers. Since the data center nodes included in each data center of the distributed architecture are based on the customer dimension and divided into groups, each data center node has an independent business processing application system, which can realize the data center nodes in the distributed architecture In the event of a failure, it still maintains high availability, disperses the processing pressure and failure risk of a single data center, and effectively reduces the scope of the failure. Moreover, Y different data centers store customer data and/or application systems, and customer data is also realized Multi-activity and/or application multi-activity can seamlessly provide services even if there is a failure, reducing the scope of the risk of failure.

Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. In this way, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

A distributed architecture, which is characterized by comprising: N data centers and M groups of data center nodes;

Each data center includes X data center nodes, each group of data center nodes includes Y data center nodes, and Y data center nodes in each group of data center nodes are located in Y different data centers; each data center node includes A database server storing customer data of Z customers and/or an application server storing an application system that processes all services of the Z customers;

Wherein, the database server storing the customer data of each customer and/or the application server storing the application system processing all the services of the Z customers are stored in one data center node, and at least Q pieces of data in the N data centers The centers are located in different cities, the Y data center nodes in each group of data center nodes include one master data center node and at least two slave data center nodes, and N, M, X, Y, Z, and Q are positive integers.
The distributed architecture of claim 1, wherein the Y data center nodes in each group of data center nodes include one master data center node, and at least one slave data center node in the same city as the master data center node A data center node and at least one slave data center node remote from the master data center node.
The distributed architecture of claim 1, wherein the customer data of the Z customers stored in the master data center node and the slave data center node are the same, and/or the stored processing of all the services of the Z customers The application system is the same.
The distributed architecture of claim 1, wherein the physical resources among the Y data center nodes are isolated from each other.
The distributed architecture according to claim 4, wherein the Y data center nodes preset one data center node for grayscale publishing.
The distributed architecture according to claim 1, wherein the physical resources in each data center node include multiple database servers and multiple application servers;

The application systems are grouped according to different application domains, and different groups do not share application servers and do not share database servers.
The distributed architecture according to any one of claims 1 to 6, wherein the distributed architecture is horizontally expanded by increasing the number of data center nodes in each group of data center nodes.
The distributed architecture according to any one of claims 1 to 6, wherein the distributed architecture is vertically expanded by increasing the computing resources of a preset data center node in each group of data center nodes; Or by temporarily allocating the computing resources in the reserved computing resource pool to preset data center nodes in each group of data center nodes, the distributed architecture is vertically expanded.
The distributed architecture according to any one of claims 1 to 6, wherein the distributed architecture further includes a global positioning system;

The global positioning system adopts a preset weighted random algorithm to manage the fragmentation strategy of the customer, and locates the data center node where the customer data is stored.
9. The distributed architecture of claim 9, wherein the global positioning system communicates with application systems in each data center node through a message bus.