WO2022037359A1

WO2022037359A1 - Configuration data access method, apparatus, and device, configuration center, and storage medium

Info

Publication number: WO2022037359A1
Application number: PCT/CN2021/108147
Authority: WO
Inventors: 邓辉超
Original assignee: 百果园技术(新加坡)有限公司; 邓辉超
Priority date: 2020-08-18
Filing date: 2021-07-23
Publication date: 2022-02-24
Also published as: CN112015744A

Abstract

A configuration data access method, apparatus, and device, a configuration center, and a storage medium. The configuration data access method is applied to a configuration management server. The configuration management server is deployed in multiple first computer rooms, and a configuration server is deployed in at least two sets of second computer rooms; each first computer room is equipped with a main database, each second computer room is equipped with a secondary database, and the main database is configured to ensure strong data consistency on the basis of a preset consistency protocol. The method comprises: detecting that a configuration data write event is triggered (401); when detected that a target main database, which corresponds to the first computer room in which same is located, is in a writable state, performing a configuration data write operation for the target main database, the main database in the writable state being determined on the basis of a preset consistency protocol (402); and once it is detected that all of the main databases except for the target main database have completed data synchronization for the present write operation, feeding back write success information (403).

Description

Configuration data access method, apparatus, device, configuration center and storage medium

This application claims the priority of a Chinese patent application with application number 202010831215.3 filed with the China Patent Office on August 18, 2020, the entire contents of which are incorporated herein by reference.

technical field

The present application relates to the field of computer technology, for example, to a configuration data access method, apparatus, device, configuration center, and storage medium.

Background technique

With the development of the microservice architecture, the configuration center has increasingly become one of the core basic components in the entire service governance architecture system. Many other components in the service governance architecture system need to rely on the configuration center to achieve features such as automatic failover, high availability, and current limiting and fuse; while the business system needs to rely on the configuration center to achieve dynamic expansion and the ability to adjust behavior when the business system is running. Therefore, the configuration center needs to have high availability and effectively ensure data consistency. At the same time, since the configuration center is the underlying component, it needs to minimize the dependence on other systems or components.

In the case of a large business deployment range (such as needing to cross countries or regions), it is even more difficult to achieve high availability and data consistency in the configuration center. The type and amount of configuration data that the configuration center needs to manage is very large, and generally needs to be stored in the corresponding database, that is, the configuration center at least depends on the external system of the database. Therefore, it is very important to achieve high availability and data consistency of the database. important. In the related art, multiple databases are generally used for data storage, including at least one master database and at least one slave database, and a certain deployment scheme is adopted to ensure high availability and data consistency of the databases. Taking the common relational database MySQL as an example, the commonly used MySQL high-availability deployment solutions in the industry include Mysql Master-Master replication manager for Mysql (MMM), Master High Availability (MHA) and MySQL Group Replication (MySQL Group Replication, MGR). However, there are some problems in using these solutions directly. Both MMM and MHA have the risk of data loss; MHA and MGA need to introduce other components or middleware to provide a unified entry, and the configuration center as the underlying component does not want to rely on other components; MMM, The high availability of MHA and MGA is aimed at the master database, and is not suitable for direct use when the slave database is deployed in a large area.

SUMMARY OF THE INVENTION

The present application provides a configuration data access method, apparatus, device, configuration center and storage medium, which can optimize the configuration data access scheme of the configuration center in the microservice architecture.

The present application provides a configuration data access method, which is applied to a configuration management server of a configuration center in a micro-service architecture, where the configuration center includes a configuration center server and a configuration center client, and the configuration center server includes the A configuration management server and a configuration server, the configuration management server is deployed in a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets preset quality requirements, and the configuration server is deployed at least In two groups of second computer rooms, the same group of second computer rooms is located in the same business area, and different groups of second computer rooms are located in different business areas, each first computer room is equipped with a master database, and each second computer room is equipped with a slave database, the said The master database and the slave database are included in the database cluster corresponding to the configuration center, and all master databases are set to ensure strong data consistency based on a preset consistency protocol;

The method includes:

triggered in response to detecting a configuration data write event;

When it is detected that the target master database corresponding to the first computer room is in a writable state, the configuration data writing operation is performed on the target master database, wherein the master database in the writable state is based on the preset consistency protocol Sure;

After it is detected that the primary databases other than the target primary database among all primary databases have completed data synchronization for this writing operation, the writing success information is fed back.

The present application provides a configuration data access method, which is applied to a configuration server of a configuration center in a microservice architecture, where the configuration center includes a configuration center server and a configuration center client, and the configuration center server includes a configuration management service and the configuration server, the configuration management server is deployed in a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets the preset quality requirements, and the configuration server is deployed in at least two In a group of second computer rooms, the same group of second computer rooms are located in the same business area, and different groups of second computer rooms are located in different business areas. Each first computer room is equipped with a master database, and each second computer room is equipped with a slave database. The database and the slave database are included in the database cluster corresponding to the configuration center, and all master databases are set to ensure strong data consistency based on a preset consistency protocol;

The method includes:

triggered in response to detecting a configuration data read event;

Detect an alternate slave database in a readable state;

The purpose slave database is selected from the candidate slave database, and the configuration data read operation is performed from the database for the purpose, wherein the purpose is the first distance between the computer room where the database is located and the second computer room where the current configuration server is located. It is less than the second distance between the computer room where the candidate secondary database except the target secondary database is located in the candidate secondary database and the second computer room where the current configuration server is located.

The present application provides a configuration data access device, which is applied to a configuration management server of a configuration center in a microservice architecture. The configuration center includes a configuration center server and a configuration center client, and the configuration center server includes the configuration center server. A configuration management server and a configuration server, the configuration management server is deployed in a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets preset quality requirements, and the configuration server is deployed at least In two groups of second computer rooms, the same group of second computer rooms is located in the same business area, and different groups of second computer rooms are located in different business areas, each first computer room is equipped with a master database, and each second computer room is equipped with a slave database, the said The master database and the slave database are included in the database cluster corresponding to the configuration center, and all master databases are set to ensure strong data consistency based on a preset consistency protocol;

The device includes:

Write event monitoring module, set to detect whether the configuration data write event is triggered;

The writing operation module is set to be triggered in response to detecting that the configuration data writing event is triggered, and in the case where it is detected that the target main database corresponding to the first computer room is in a writable state, write configuration data to the target main database operations, wherein the master database in a writable state is determined based on the preset consistency protocol;

The write operation feedback module is configured to feed back write success information after detecting that the master databases in all the master databases except the target master database have completed data synchronization for this write operation.

The present application provides a configuration data access device, which is applied to a configuration server of a configuration center in a microservice architecture, where the configuration center includes a configuration center server and a configuration center client, and the configuration center server includes a configuration management service and the configuration server, the configuration management server is deployed in a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets the preset quality requirements, and the configuration server is deployed in at least two In a group of second computer rooms, the same group of second computer rooms are located in the same business area, and different groups of second computer rooms are located in different business areas. Each first computer room is equipped with a master database, and each second computer room is equipped with a slave database. The database and the slave database are included in the database cluster corresponding to the configuration center, and all master databases are set to ensure strong data consistency based on a preset consistency protocol;

The device includes:

The read event triggers the detection module, which is set to detect whether the configuration data read event is triggered;

a state detection module, configured to detect an alternate slave database in a readable state in response to detecting that the configuration data read event is triggered;

The read operation module is set to select the purpose from the database from the alternative database, and perform a configuration data read operation from the database for the purpose, wherein the purpose is from the computer room where the database is located and the current configuration server is located. The first distance of the second computer room is smaller than the second distance between the computer room where the candidate secondary database except the destination secondary database is located in the candidate secondary database and the second computer room where the current configuration server is located.

The present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the above-mentioned configuration data access method when the computer program is executed.

The application provides a configuration center in a micro-service architecture, including a configuration center server and a configuration center client, the configuration center server includes a configuration management server and a configuration server, and the configuration management server is deployed in multiple In a first computer room, the network quality between the multiple first computer rooms meets the preset quality requirements, the configuration server is deployed in at least two groups of second computer rooms, and the same group of second computer rooms is located in the same service area, Different groups of second computer rooms are located in different business areas, each first computer room is equipped with a master database, and each second computer room is equipped with a slave database, the master database and the slave database are included in the database cluster corresponding to the configuration center , all master databases are set to ensure strong data consistency based on preset consistency protocols;

The configuration management server is set to perform configuration data access based on the above-mentioned configuration data access method applied to the configuration management server;

The configuration server is configured to perform configuration data access based on the above-mentioned configuration data access method applied to the configuration server.

The present application provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned configuration data access method is implemented.

Description of drawings

1 is a scenario architecture diagram of an application scenario to which a configuration data access method provided by an embodiment of the present application is applicable;

FIG. 2 is a schematic diagram of a configuration center deployment solution provided by an embodiment of the present application;

3 is a schematic diagram of a database cluster deployment solution provided by an embodiment of the present application;

4 is a schematic flowchart of a configuration data access method provided by an embodiment of the present application;

5 is a schematic flowchart of another configuration data access method provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of an implementation manner of an intelligent data source provided by an embodiment of the present application;

FIG. 7 is a structural block diagram of a configuration data access device provided by an embodiment of the present application;

8 is a structural block diagram of another configuration data access device provided by an embodiment of the present application;

FIG. 9 is a structural block diagram of a computer device according to an embodiment of the present application.

detailed description

The present application will be described below with reference to the accompanying drawings and embodiments.

This embodiment mainly improves the configuration center in the microservice architecture. The basic idea of microservices is to consider creating applications around business domain components that can be independently developed, managed and accelerated, using microservices cloud architecture and platforms in decentralized components, enabling deployment, management and service functions Interaction just got easier. With the increasing complexity of application functions in the microservice architecture, the configuration of the program is increasing, such as the switch of various functions, the configuration of parameters, and the address of the server, etc., and the expectations for configuration are also getting higher and higher, such as configuration After modification, it takes effect in real time, grayscale release, sub-environment or sub-cluster management configuration, perfect authority mechanism and audit mechanism, etc., adopt a distributed development model, so that the mutual reference between projects increases with the continuous increase of services. The complexity of calling each other between projects increases exponentially, and the workload continues to increase each time a new project is put into production or online, so it is necessary to refer to the configuration center for governance.

FIG. 1 is a scenario architecture diagram of an application scenario to which a configuration data access method provided by an embodiment of the present application is applicable. Referring to FIG. 1 , the application scenario may include a configuration center (Config Center) 100 , a database (DataBase, DB) 200 and a business system 300 . The configuration center 100 includes a configuration center server 101 and a configuration center client (Config Client) 102, the configuration center server 101 includes a configuration management service (Config Admin Service) end 1011 and a configuration service (Config Service) end 1012, and the database 200 Including the master database 201 and the slave database 202. The configuration center 100 can be understood as a complete set of systems that provide distributed configuration services. The distributed configuration service can include functions such as configuration modification, publishing, and push. The configuration management server 1011, the configuration server 1012, and the configuration center client 102 can be understood as Submodules in the configuration center 100 are configured. Among them, the configuration server 1012 is generally a read-only server, providing configuration pull and push services for clients; the configuration management server 1011 can provide configuration management services, including configuration viewing, modification, and publishing functions; The configuration center client 102 can provide a software development kit (Software Development Kit, SDK) used by the business system, and interact with the configuration server 1012 to implement configuration pull and push functions. The configuration center 100 needs to have high availability and effectively ensure data consistency. At the same time, since the configuration center 100 is a bottom-level component, it needs to minimize the dependence on other systems or components. The configuration center 100 needs to manage the types and data volumes of the configuration data. It is very huge and generally needs to be stored in the corresponding database 200 , that is, the configuration center at least depends on the external system of the database 200 . Considering factors such as data reliability, a master database 201 and a slave database 202 are generally provided. The configuration management server 1011 can perform write operations such as configuration modification for the master database 201. The master database 201 and the slave database 202 perform data synchronization and configure services. The terminal 1012 may perform a configuration read operation from the database 202 to provide configuration data to the configuration center client 102 .

With the continuous expansion of the business scope, the business system forms multiple business areas (for example, divided by continents, countries, regions or cities, etc.), and it is necessary to deploy equipment equipped with multiple sub-modules or databases of the configuration center in a large geographical scope. (such as servers, etc.), the computer rooms where the servers are stored are widely distributed, and the communication distance between each two computer rooms is also greatly increased, so the network environment becomes more and more unreliable, and the reliability and availability of the configuration center implementation plan are more and more difficult to guarantee. .

In related technologies, taking common MySQL as an example, commonly used high-availability deployment solutions include MMM, MHA, and MGR. Among them, MMM is a set of script programs that support dual-master failover and dual-master daily management. Since MMM cannot fully guarantee data consistency, MMM is only suitable for scenarios where data consistency requirements are not very high; MHA is MySQL A relatively mature solution in terms of high availability, the software consists of two parts, including a management node (MHA Manager) and a data node (MHA Node). MHA Manager will regularly detect the master (master) node in the cluster, when the master node When a node fails, it can automatically promote the slave (slave) node that stores the latest data to the new master node, and then redirect all other slave nodes to the new master node; MGR is a replication plug-in based on MySQL architecture, using The Paxos protocol implements data consistency among multiple nodes, and MGR supports single-master mode and multi-master mode. In the above three schemes, both MMM and MHA have the risk of data loss; MHA and MGA need to introduce other components (such as virtual IP address (Virtual IP Address, VIP) components) or database middleware to provide a unified entry, relying on additional operations. Maintenance tools or middleware achieve high availability, while database middleware relies on the configuration center to achieve high availability. Therefore, circular dependencies will be formed, and the configuration center, as the underlying component, does not want to depend on other components; in addition, the above three The high availability of the scheme is aimed at the master database, and is not suitable for direct use when the slave database is deployed in a larger area.

The configuration center in the embodiment of the present application includes a configuration center server and a configuration center client, the configuration center server includes a configuration management server and a configuration server, and the configuration management server is deployed in a plurality of first computer rooms, The network quality between the plurality of first computer rooms meets the preset quality requirements, the configuration server is deployed in at least two groups of second computer rooms, the same group of second computer rooms is located in the same business area, and different groups of second computer rooms are located in the same business area. In different business areas, each first computer room is equipped with a master database, and each second computer room is equipped with a slave database. The master database and the slave database are included in the database cluster corresponding to the configuration center. Set up a consistency protocol to ensure strong data consistency. Among them, strong consistency can mean that any read operation can read the last written value of a data, or, at any time, the data in all nodes is the same, for example, for a relational database, it is required to update Subsequent access to the past data can be seen.

Exemplarily, the number of the first computer room is not limited, that is, the number of deployments of the configuration management server is not limited, which can be set according to the actual situation (such as the number of business areas). For example, the number of the first computer room can be set as Three, one configuration management server can be integrated into computer equipment such as a server in the corresponding first computer room. Each first computer room is equipped with a main database, that is, the number of the main databases and the number of the first computer rooms can be equal, that is, the number of the main databases can be equal to the deployment number of the configuration management server. The configuration management server can access configuration data for its corresponding main database, and the device where the main database is located and the device where the configuration management server is located can be the same. The types of the master database and the slave database are generally the same. This application does not limit the types of the master database and the slave database. For example, the types of the master database and the slave database can be relational databases such as MySQL, or can be a remote dictionary service (Remote dictionary server). , Redis) and other non-relational databases. All master databases ensure strong data consistency based on a preset consistency protocol. Therefore, the network quality between multiple first computer rooms needs to meet the preset quality requirements, and the network quality between multiple first computer rooms can meet the preset quality requirements. The network quality between any two first computer rooms in all the first computer rooms meets the preset quality requirements, or there may be another first computer room corresponding to each first computer room in all the first computer rooms and each of the first computer rooms The network quality between a computer room meets the preset quality requirements. Among them, the network quality can be measured by indicators such as packet loss rate or round-trip time (RTT). Connection, to ensure that the network communication between each other is reliable. The preset consistency protocol may be, for example, the Paxos protocol, or other protocols that can ensure strong data consistency between databases. Through the preset consistency protocol, one or more master nodes can be elected in multiple master databases to be set for data writing, that is, it can be guaranteed that one or more master nodes can be set to write data, and other master databases automatically become The slave node of the master node, when writing configuration data to the master node, the master node synchronizes the written configuration data to the slave node, that is, the slave node can passively write the configuration data. After the data write operations are completed, the preset consistency protocol will confirm the success of the write operation, thereby ensuring strong data consistency between the primary databases. In MMM and MHA, data synchronization between master-slave nodes or between dual-master nodes is asynchronous, that is to say, during a write operation, as long as one database completes the write, the next write can be performed. , if there is a problem with the database at this time, and other databases have not been synchronized in time, it will cause data loss. However, in the embodiment of the present application, strong data consistency is ensured between multiple master databases based on a preset consistency protocol, and data loss due to a problem in one of the master databases will not occur.

Exemplarily, the slave database is deployed in the business area. The division rules of business areas and the number of business areas can be determined according to the actual business scope and other related factors. A group of second computer rooms can be deployed in each business area. The number of second computer rooms in different groups can be the same or different. The application does not limit the number of second computer rooms in each group of second computer rooms. The number of second computer rooms is not limited, that is, the number of configuration servers deployed is not limited, and one configuration server can be integrated into computer equipment such as servers in the corresponding second computer room. Each second computer room is equipped with a slave database, and the number of slave databases can be equal to the number of second computer rooms, that is, the number of slave databases can be equal to the deployment quantity of the configuration server, and the configuration server can target its corresponding slave database. For configuration data access, the device where the database is located and the device where the configuration server is located can be the same.

The following is a schematic description of the configuration center deployment scheme and the database cluster deployment scheme. Take the master database and the slave database as MySQL databases, and the default consistency protocol as the Paxos protocol. The first computer room corresponds to a business area, a group of second computer rooms is deployed in each business area, and the number of second computer rooms in each group of second computer rooms is three.

FIG. 2 is a schematic diagram of a configuration center deployment scheme provided by an embodiment of the present application. As shown in FIG. 2 , the portal (Portal) can be understood as a human-computer interaction interface operated by staff, and the staff can use the Portal to manage services (Configuration) according to the configuration. The address information of the device where the Admin Service) end is located, such as IP address and port number, to call the Config Admin Service to perform access operations such as writing configuration data, and the Config Admin Service end can write the configuration data to the corresponding master database. Among them, three main databases can form an MGR group. The MGR group realizes strong data consistency based on the Paxos protocol. In this deployment scheme, the Portal and Config Admin Service mainly provide configuration management, writing functions, and reading and writing the main database. Requires global deployment. The Config Service side is set to implement configuration pull and real-time push for the business, which requires high availability and requires global deployment. The Config Service side can select the nearest slave database to read configuration data, and each business area is set with a corresponding slave database. , such as three Asian slave databases, three European slave databases, and three Americas slave databases.

Figure 3 is a schematic diagram of a database cluster deployment solution provided by an embodiment of the present application. As shown in Figure 3, an MGR Group consists of three nodes (Nodes), that is, three main databases, and the three nodes are elected through the Paxos protocol A Master node is set to write data, and the data of the three nodes is kept strongly consistent through the Paxos protocol. The configuration management server can perceive the role changes of the three nodes of the MGR Group and automatically select the Master node for configuration writing. For implementation details, please refer to the related content below. In order to take into account the data synchronization performance and availability of the Paxos protocol, the three nodes are deployed in three computer rooms in the same city. Slave databases are deployed on three continents around the world, and three computer rooms (second computer rooms) are deployed on each continent. Asynchronous replication (Replicate) can be used to achieve eventual consistency between the slave database and the master database. The configuration server of each second computer room preferentially reads the slave database in the computer room, and downgrades to read other slave databases in the same area when the slave database in the computer room is unavailable, so as to realize the mutual backup of the slave databases, so as to achieve high availability. The implementation details can be See related content below.

The above is only an example, the type and quantity of the master database, the type and quantity of the slave database, the type of the preset consistency protocol, the division method of the business area, the number of the first computer room, and the second computer room in each group of the second computer room. The number of computer rooms, etc., can be set according to actual needs.

4 is a schematic flowchart of a configuration data access method provided by an embodiment of the present application. The method is applied to a configuration management server of a configuration center in a microservice architecture, and can be executed by a configuration data access device, which can be configured by software and/or Or hardware implementation, generally can be integrated in computer equipment such as servers. As shown in Figure 4, the method includes:

Step 401, it is detected that a configuration data write event is triggered.

Exemplarily, the configuration data writing event here may be a configuration data writing event for the master database, and the triggering condition of the event may be set according to actual requirements. For example, when a staff member needs to perform configuration addition, configuration modification, configuration deletion or other configuration data change operations, the above-mentioned addition, modification and deletion can be regarded as write operations. The configuration data writing to the main database is realized by calling the configuration management server. When the configuration management server receives the configuration data writing request sent by the configuration management device, it can be considered that the configuration data writing has been detected. event is fired. This application does not limit the number of configuration management devices, which can be one or more. When the number is more than one, it can be the same as the number of the first computer room. The configuration management devices can be set in the first computer room or in Other locations can communicate with the device where the configuration management server is located.

Step 402: If it is detected that the target master database corresponding to the first computer room is in a writable state, perform a configuration data writing operation for the target master database, wherein the master database in the writable state is based on the preset consistency. Agreement is confirmed.

Exemplarily, multiple master databases may elect a master database as a master node based on a preset consistency protocol, and the master database as a master node is in a writable state. The number of master nodes may be one or more, and this application does not limit the number of master nodes. If the target master database corresponding to the first computer room where the current configuration management server is located is the master node, the target master database is in a writable state, and the current configuration management server can detect that the target master database is writable, and then can target the target master database. The primary database performs configuration data write operations. If the target master database corresponding to the first computer room where the current configuration management server is located is not the master node, other configuration management servers will detect the writable master database, thereby realizing highly reliable configuration data writing.

Step 403: After detecting that the primary database other than the target primary database has completed the data synchronization for this writing operation, feedback the writing success information.

Exemplarily, in the process of writing configuration data to the target master database, since all master databases ensure strong data consistency based on a preset consistency protocol, the target master database will synchronize the written configuration data to other master databases. In the database, after detecting that other primary databases have completed data synchronization for this write operation, it can confirm that the configuration data to be written has been successfully written to all primary databases, and can feedback the successful writing information to the configuration management device. for subsequent operations.

In the configuration data access method provided in the embodiment of this application, the configuration center in the micro-service architecture includes a configuration center server and a configuration center client, the configuration center server includes a configuration management server and a configuration server, and the configuration management server is deployed in In multiple first computer rooms, each first computer room is equipped with a master database. All master databases ensure strong data consistency based on a preset consistency protocol. The network quality between the first computer rooms meets the preset quality requirements, which can ensure that the first computer room is At the same time, it can also ensure the synchronization performance between the main databases and the high availability of the main database. The configuration server is deployed in at least two sets of second computer rooms, and the same group of second computer rooms are located in the same business area. Different groups of second computer rooms are located in different business areas, and each second computer room is equipped with a slave database. The master database and the slave database are included in the database cluster corresponding to the configuration center. Based on the architecture of the configuration center, the configuration management server detects that the After the configuration data write event is triggered, it actively detects whether the target master database corresponding to the first computer room is in a writable state. If the target master database corresponding to the first computer room is in a writable state, the configuration data is written to the target master database. The main database in the writable state is determined based on the preset consistency protocol, that is, the master node defined in the preset consistency protocol, so as to ensure strong data consistency between the master databases based on this protocol, and detect After the data synchronization for this write operation is completed in the main database other than the target main database, the write success information is fed back, so that the intelligent detection of the writable main database and the configuration data can be realized in the configuration center without relying on other components. High-reliability write operations, and all primary databases are based on preset consistency protocols to ensure high availability and data consistency of the primary databases, avoiding business impact due to data loss.

In some embodiments, before the detecting that the configuration data writing event is triggered, the method further includes: acquiring topology structure information of the database cluster, wherein the topology structure information includes the corresponding relationship between the computer room and the database; The topological structure information is queried in the target master database corresponding to the first computer room where it is located. The advantage of this setting is that it is convenient for the configuration management server to quickly and accurately confirm the target master database corresponding to itself.

Exemplarily, the topology information can be included in a database topology configuration file, and the application does not limit the format of the configuration file, for example, it can be in a JS Object Notation (JavaScript Object Notation, JSON) format, such as a database topology configuration file. Can be recorded as datasource.json. The topology information can include a list of master databases and a list of slave databases. The master database list may include the IP and port number of each master database, and the slave database list may include computer room information and location information (such as local area information) corresponding to each slave database. For the topology information read by the configuration management server, it may only include the list of the main database. The configuration management server can confirm the IP and port number of its own device, and then compare its own IP and port number with the IP and port number in the main database list, and find the highest similarity (if the device where the configuration management server is located and If the device where the corresponding master database is located is the same, then the master database with the same IP and port number is determined as its corresponding target master database.

In some embodiments, after the querying the target master database corresponding to the first computer room according to the topology information, the method further includes: establishing a first connection with the target master database; detecting the target master database Being in a writable state includes: attempting to write first data to the target primary database based on the first connection, and determining that the target primary database is in a writable state if it is detected that the writing of the first data is successful. The advantage of this setting is that after determining the corresponding target master database, a connection can be established with the target master database. When configuration data access is required, there is no need to temporarily establish a connection, which improves the efficiency of configuration data access. In addition, by trying to write The data method can accurately judge the status of the target master database.

In some embodiments, after establishing the first connection with the target master database, the method further includes: if a change in the topology structure information is monitored, determining a new target master database according to the changed topology structure information; The new target master database establishes a second connection. The advantage of this setting is that if the topology information changes, the corresponding target master database may also change, and the first connection with the original target master database can be disconnected, and the second connection with the new target master database can be established. When configuration data access is required in the future, there is no need to temporarily change the connection, which improves the efficiency of configuration data access.

In some embodiments, the method further includes: detecting that a configuration data read event is triggered; detecting a candidate master database in a readable state; selecting a target master database from the candidate master databases, and targeting the purpose The main database performs a configuration data read operation, wherein the first distance between the computer room where the target main database is located and the first computer room where the current configuration management server is located is smaller than the first computer room where the other candidate master databases are located and the first computer room where the current configuration management server is located. the second distance. The advantage of this setting is that the configuration management server may also need to read the configuration data. When the configuration management server needs to read the configuration data in the main database, it can select the nearest main database for the read operation. Generally, The main database of the purpose is the main database of the same equipment or the same computer room as the current configuration management server. When the nearest main database is abnormal, the nearest main database can also be selected from the main databases in the readable state for reading. operate. The detection method may be one by one according to the distance between the computer room where the main database is located and the first computer room where the current configuration management server is located, or may be batch detection. The application does not limit the detection method.

5 is a schematic flowchart of another configuration data access method provided by an embodiment of the present application. The method is applied to a configuration server of a configuration center in a microservice architecture, and can be executed by a configuration data access device, which can be implemented by software and/or Or hardware implementation, generally can be integrated in computer equipment such as servers. As shown in Figure 5, the method includes:

Step 501, it is detected that a configuration data read event is triggered.

Exemplarily, the configuration data reading event here may be an event for reading configuration data from a database, and the triggering condition of the event may be set according to actual requirements. For example, when the business system needs to obtain configuration data, it can send a configuration data acquisition request to the configuration server through the configuration center client. The configuration server needs to read the configuration data from the database. When the configuration server receives the request, It can be considered that the configuration data read event is triggered; for another example, the configuration server can push the configuration data to the business system according to the preset push rules (such as timing or real-time push), and when it is detected that the configuration data needs to be pushed, the configuration service The client needs to read the configuration data from the database, and it can be considered that the configuration data read event is triggered. Generally, a configuration server can correspond to one or more configuration center clients, that is, a configuration server can provide configuration pull or push services for one or more business systems.

Step 502: Detect a candidate slave database in a readable state.

Exemplarily, the detection range may be a service area corresponding to the currently configured server, and detection operations may be reduced. To further ensure availability, the detection range may also be multiple service areas. The configuration server in the embodiment of the present application can automatically detect the candidate slave database in a readable state, so as to facilitate the subsequent selection of the optimal slave database to perform the read operation of configuration data. The detection method may be one by one according to the distance between the computer room where the database is located and the computer room where the current configuration server is located, or may be batch detection, which is not limited in this application.

Step 503, select the purpose from the database from the alternative database, and perform a configuration data read operation from the database for the purpose, wherein the purpose is from the computer room where the database is located and the second computer room where the current configuration server is located. The first distance is smaller than the second distance between the computer room where the other candidate slave databases are located and the second computer room where the current configuration server is located.

Exemplarily, the closest slave database may be selected from the candidate slave databases in the readable state to read the configuration data. Generally, the slave database that is in the same device or room as the configuration server is the closest. If the slave database is abnormal and is not in a readable state, other slave databases in the same business area can be selected to read the configuration data. Among them, the granularity of the distance can be freely set, which can be the distance of the actual geographic coordinates, or it can be a city or a region. For example, if the granularity of the distance is a city, the distance between the computer rooms in the same city can be considered as 0, and the distance between the computer rooms in two adjacent cities can be considered as 1 and so on.

In the configuration data access method provided in the embodiment of this application, the configuration center in the micro-service architecture includes a configuration center server and a configuration center client, the configuration center server includes a configuration management server and a configuration server, and the configuration management server is deployed in In multiple first computer rooms, each first computer room is equipped with a master database. All master databases ensure strong data consistency based on a preset consistency protocol. The network quality between the first computer rooms meets the preset quality requirements, which can ensure that the first computer room is At the same time, it can also ensure the synchronization performance between the main databases and the high availability of the main database. The configuration server is deployed in at least two sets of second computer rooms, and the same group of second computer rooms are located in the same business area. Different groups of second computer rooms are located in different business areas. Each second computer room is equipped with a slave database. The master database and the slave database are included in the database cluster corresponding to the configuration center. Based on the architecture of the configuration center, the configuration server detects the configuration After the data read event is triggered, it actively detects the candidate slave database in the readable state, and selects the candidate slave database closest to the current configuration server to read the configuration data, realizes the mutual backup of the slave databases, and ensures the configuration center. high availability.

In some embodiments, before the detecting that the configuration data read event is triggered, the method further includes: acquiring topology structure information of the database cluster, wherein the topology structure information includes information about the correspondence between the computer room and the database, And the corresponding relationship information between the database and the business area; according to the topology information, query the multiple slave databases contained in the target business area corresponding to the second computer room; The detection of the candidate slave database in a readable state includes: A candidate slave database in a readable state is detected from the queried multiple slave databases. The advantage of this setting is that it is convenient for the configuration server to quickly and accurately confirm the business area corresponding to itself, and quickly detect the candidate slave database in the readable state from the business area.

As mentioned above, the topology information can be included in the database topology configuration file, and the topology information in the configuration server can be the same as the topology information in the configuration management server, that is, the configuration server and the configuration management server belong to The same database topology configuration file may exist in all devices, which facilitates the unified writing, unified update and unified distribution of the database topology configuration file. Optionally, the topology information in the configuration server can also only include the slave database list. The configuration server can confirm the region and computer room to which it belongs, and then query the slave database that is the same as its own region and computer room in the slave database list.

In some embodiments, after the querying, according to the topology information, the multiple slave databases included in the target business area corresponding to the second computer room, the method further includes: separately establishing a third slave database with the queried multiple slave databases. connecting; the detecting a candidate slave database in a readable state from the queried multiple slave databases includes: trying to read the second data from the multiple queried slave databases based on the third connection. , if it is detected that the second data is successfully read, it is determined that the corresponding slave database is in a readable state, and the determined slave database is used as an alternate slave database. The advantage of this setting is that after querying the corresponding slave database, you can establish a connection with the slave database. When you need to access configuration data, you do not need to temporarily establish a connection, which improves the efficiency of configuration data access. In addition, by trying to read data In this way, the state of the slave database can be accurately judged. When establishing the third connection with the slave database, the third connection with the slave database in the same device or in the same computer room as the current configuration server can be established first. When a read operation is required, if the slave database is detected based on the third connection When the database is not in a readable state, a third connection is established with other slave databases, and an alternate slave database in a readable state is detected.

In some embodiments, after respectively establishing third connections with the multiple slave databases that have been queried, the method further includes: if it is monitored that the topology information changes, determining a new query query according to the changed topology information. The multiple slave databases obtained by the new query respectively establish a fourth connection with the multiple slave databases obtained by the new query. The advantage of this setting is that if the topology information changes, the corresponding slave database may also change, so the third connection with the original slave database can be disconnected, and a fourth connection can be established with the newly queried slave database. When the configuration data needs to be read later, there is no need to temporarily change the connection, which improves the efficiency of configuration data reading.

In some embodiments, the configuration data access method applied to the configuration management server and the configuration data access method applied to the configuration server can be completed by the same program code, and the program can intelligently determine that the configuration data write is currently triggered. The event is still a configuration data read event. Whether the configuration data read event is for the master database or the slave database can be judged according to the received request type or Structured Query Language (SQL) statement.

Exemplarily, in this embodiment of the present application, the above program may be referred to as an intelligent data source. FIG. 6 is a schematic diagram of an implementation manner of an intelligent data source provided in an embodiment of the present application. As shown in FIG. 6 , datasource.json is the above-mentioned Database topology configuration file. DataSourceConfigSyncer configures a listener for the data source, which is responsible for monitoring changes in datasource.json and notifying the DataSourceManager to rebuild the smart data source. SmartDataSource is the implementation class of smart data source, which provides a unified data access interface to the upper layer. It contains the write data source MultiWriteDataSource and the read data source MultiReadDataSource, and route to the corresponding data source according to the type of SQL.

MultiWriteDataSource is the encapsulation of write data source, which contains multiple (for example, 3, the same as the number of main database) WriteDataSource, MultiWriteDataSource will select a writable WriteDataSource to write, and whether it is writable is determined by write heartbeat (WriteHeartbeat) The write SQL is periodically sent for detection. In this embodiment of the present application, the single-master mode in the MGR may be adopted. In this mode, only one node can write at any time.

MultiReadDataSource is the encapsulation of read data source, which contains multiple (for example, a business area deploys 3 slave databases, the number here can be 3) ReadDataSource, MultiReadDataSource will read from all healthy (that is, in readable state) ReadDataSource Select an optimal ReadDataSource for read operations. The health status is detected by periodically sending read SQL through the read heartbeat (ReadHeartbeat), and the selection of the optimal ReadDataSource is based on the priority field indication (select the smallest priority value). The priority is sorted by distance, that is, The smaller the distance, the smaller the corresponding priority value.

Through the above configuration server and MySQL cluster deployment scheme and intelligent data source design, high availability and eventual consistency of configuration in global deployment and unreliable network environment can be achieved, and system autonomy can be achieved, and no faults occur. Failover can be achieved without any external intervention (that is, without relying on other components such as middleware, and without human intervention) to achieve high availability of the master database and high availability of slave databases in multiple business areas.

7 is a structural block diagram of a configuration data access device provided by an embodiment of the present application. The device is applied to a configuration management server of a configuration center in a microservice architecture, and can be implemented by software and/or hardware, and can generally be integrated in a server, etc. In the computer equipment, configuration data access may be performed by executing a configuration data access method. As shown in Figure 7, the device includes:

The write event monitoring module 701 is set to detect whether the configuration data write event is triggered; the write operation module 702 is set to, after detecting that the configuration data write event is triggered, if the target host corresponding to the first computer room where it is located is detected If the database is in a writable state, the configuration data writing operation is performed on the target master database, wherein the master database in the writable state is determined based on the preset consistency protocol; the write operation feedback module 703 is set to detect After the master database other than the target master database completes the data synchronization for this writing operation, the writing success information is fed back.

In the configuration data access device provided by the embodiment of the present application, the configuration center in the micro-service architecture includes a configuration center server and a configuration center client, the configuration center server includes a configuration management server and a configuration server, and the configuration management server is deployed in multiple In the first computer room, each first computer room is equipped with a master database. All the master databases ensure strong data consistency based on the preset consistency protocol. The network quality between the first computer rooms meets the preset quality requirements, which can ensure the first computer room. At the same time, it can also ensure the synchronization performance between the main databases and the high availability of the main database. The configuration server is deployed in at least two groups of second computer rooms. The same group of second computer rooms are located in the same business area, but different The second computer room of the group is located in different business areas. Each second computer room is equipped with a slave database. The master database and the slave database are included in the database cluster corresponding to the configuration center. Based on the architecture of the configuration center, the configuration management server detects the configuration After the data write event is triggered, it actively detects whether the target master database corresponding to the first computer room is in a writable state. If the target master database corresponding to the first computer room is in a writable state, the configuration data is written to the target master database. operation, and the master database in the writable state is determined based on the preset consistency protocol, that is, the master node defined in the preset consistency protocol, so as to ensure strong data consistency between master databases based on this protocol. After the master database other than the target master database completes the data synchronization for this write operation, the write success information is fed back, so that the intelligent detection of the writable master database and the high level of configuration data can be realized in the configuration center without relying on other components. Reliable write operations, and all primary databases ensure high availability and data consistency of the primary databases based on preset consistency protocols, avoiding business impact due to data loss.

8 is a structural block diagram of another configuration data access device provided by an embodiment of the present application. The device is applied to a configuration server of a configuration center in a microservice architecture, and can be implemented by software and/or hardware, and can generally be integrated in a server, etc. In the computer equipment, configuration data access may be performed by executing a configuration data access method. As shown in Figure 8, the device includes:

The read event trigger detection module 801 is configured to detect whether the configuration data read event is triggered; the state detection module 802 is configured to detect an alternative slave database in a readable state after detecting that the configuration data read event is triggered The read operation module 803 is set to select the purpose from the database from the alternative database, and carry out the configuration data reading operation from the database for the purpose, wherein, the purpose is from the computer room where the database is located and the current configuration service. The first distance of the second computer room where the terminal is located is smaller than the second distance between the computer room where the other candidate slave databases are located and the second computer room where the current configuration server is located.

In the configuration data access device provided in the embodiment of this application, the configuration center in the micro-service architecture includes a configuration center server and a configuration center client, the configuration center server includes a configuration management server and a configuration server, and the configuration management server is deployed in In multiple first computer rooms, each first computer room is equipped with a master database. All master databases ensure strong data consistency based on a preset consistency protocol. The network quality between the first computer rooms meets the preset quality requirements, which can ensure that the first computer room is At the same time, it can also ensure the synchronization performance between the main databases and the high availability of the main database. The configuration server is deployed in at least two sets of second computer rooms, and the same group of second computer rooms are located in the same business area. Different groups of second computer rooms are located in different business areas. Each second computer room is equipped with a slave database. The master database and the slave database are included in the database cluster corresponding to the configuration center. Based on the architecture of the configuration center, the configuration server detects the configuration After the data read event is triggered, it actively detects the candidate slave database in the readable state, and selects the candidate slave database closest to the current configuration server to read the configuration data, realizes the mutual backup of the slave databases, and ensures the configuration center. high availability.

The embodiments of the present application provide a computer device, and the configuration data access apparatus provided by the embodiments of the present application can be integrated into the computer device. FIG. 9 is a structural block diagram of a computer device according to an embodiment of the present application. The computer device 900 includes a memory 901, a processor 902, and a computer program that is stored in the memory 901 and can run on the processor 902. When the processor 902 executes the computer program, the configuration data access method provided by the embodiments of the present application is implemented. .

The embodiments of the present application further provide a storage medium containing computer-executable instructions, where the computer-executable instructions, when executed by a computer processor, are used to execute the configuration data access method provided by the embodiments of the present application.

Embodiments of the present application further provide a configuration center in a micro-service architecture, including a configuration center server and a configuration center client, the configuration center server includes a configuration management server and a configuration server, and the configuration management server deploys In a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets the preset quality requirements, the configuration server is deployed in at least two groups of second computer rooms, and the same group of second computer rooms is located in the same service different groups of second computer rooms are located in different business areas, each first computer room is equipped with a master database, and each second computer room is equipped with a slave database, the master database and the slave database are included in the database corresponding to the configuration center In the cluster, all main databases ensure strong data consistency based on a preset consistency protocol; the configuration management server performs access operations such as configuration data writing based on the configuration data access method described in the embodiment of the present application; the configuration server is based on the The configuration data access method described in the embodiments of the present application performs access operations such as configuration data reading.

The configuration data access device, device, storage medium, and configuration center provided in the above embodiments can execute the configuration data access method provided by the embodiment of the present application, and have corresponding functional modules and effects for executing the method. For technical details not described in detail in the foregoing embodiments, reference may be made to the configuration data access method provided by the corresponding embodiment of the present application.

Claims

A configuration data access method is applied to a configuration management server of a configuration center in a microservice architecture, the configuration center includes a configuration center server and a configuration center client, and the configuration center server includes the configuration management server and a configuration server, the configuration management server is deployed in a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets the preset quality requirements, and the configuration server is deployed in at least two sets of second computer rooms. In the computer room, the same group of second computer rooms are located in the same business area, and different groups of second computer rooms are located in different business areas. Each first computer room is equipped with a master database, and each second computer room is equipped with a slave database. The slave database is included in the database cluster corresponding to the configuration center, and all master databases are set to ensure strong data consistency based on a preset consistency protocol;

The method includes:

triggered in response to detecting a configuration data write event;

When it is detected that the target master database corresponding to the first computer room is in a writable state, the configuration data writing operation is performed on the target master database, wherein the master database in the writable state is based on the preset consistency protocol Sure;

After it is detected that the main databases other than the target main database among all the main databases have completed data synchronization for this write operation, the write success information is fed back.
The method according to claim 1, before the detecting that the configuration data write event is triggered, further comprising:

acquiring topology information of the database cluster, wherein the topology information includes information on the correspondence between the computer room and the database;

The target master database corresponding to the first computer room where it is located is queried according to the topology structure information.
The method according to claim 2, after querying the target master database corresponding to the first computer room according to the topology structure information, further comprising:

establishing a first connection with the target master database;

The detected target master database corresponding to the first computer room is in a writable state, including:

Based on the first connection attempt to write the first data to the target primary database, in the case that the writing of the first data is detected to be successful, it is determined that the target primary database is in a writable state.
The method according to claim 3, after establishing the first connection with the target master database, further comprising:

In the case of monitoring the change of the topology structure information, determine a new target master database according to the changed topology structure information;

A second connection is established with the new target master database.
The method of claim 1, further comprising:

It is detected that the configuration data read event is triggered;

Detects an alternate primary database in a readable state;

The target main database is selected from the candidate main database, and the configuration data reading operation is performed on the target main database, wherein the first computer room where the target main database is located and the first computer room where the current configuration management server is located is located. The distance is smaller than the second distance between the computer room where the candidate primary databases other than the target primary database are located in the candidate primary database and the first computer room where the current configuration management server is located.
A configuration data access method is applied to a configuration server of a configuration center in a microservice architecture, the configuration center includes a configuration center server and a configuration center client, the configuration center server includes a configuration management server and the configuration center server A configuration server, the configuration management server is deployed in a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets a preset quality requirement, and the configuration server is deployed in at least two sets of second computer rooms In the same group of second computer rooms are located in the same business area, different groups of second computer rooms are located in different business areas, each first computer room is equipped with a master database, each second computer room is equipped with a slave database, the master database and the The slave database is included in the database cluster corresponding to the configuration center, and all master databases are set to ensure strong data consistency based on a preset consistency protocol;

The method includes:

triggered in response to detecting a configuration data read event;

Detect an alternate slave database in a readable state;

The purpose slave database is selected from the candidate slave database, and the configuration data read operation is performed from the database for the purpose, wherein the purpose is the first distance between the computer room where the database is located and the second computer room where the current configuration server is located. It is less than the second distance between the computer room where the candidate slave databases other than the destination slave database are located in the candidate slave database and the second computer room where the current configuration server is located.
The method according to claim 6, before the detecting that the configuration data read event is triggered, further comprising:

Obtaining the topology structure information of the database cluster, wherein the topology structure information includes the information of the correspondence between the computer room and the database, and the information of the correspondence between the database and the business area;

Query multiple slave databases included in the target business area corresponding to the second computer room according to the topology information;

The detected candidate slave database in a readable state includes:

The candidate slave database in the readable state is detected from the plurality of slave databases queried.
The method according to claim 7, after the multiple slave databases included in the target business area corresponding to the second computer room where the query is based on the topology structure information, further comprising:

respectively establishing a third connection with the queried multiple slave databases;

The detection of the candidate slave database in a readable state from the plurality of slave databases queried includes:

Based on the third connection attempt to read the second data from the plurality of queried slave databases, respectively, in the case of detecting that the second data is successfully read, determine the second data that has been successfully read The corresponding slave database is in a readable state, and the determined slave database is used as the candidate slave database.
The method according to claim 8, after the third connection is respectively established with the plurality of queried slave databases, further comprising:

In the case of monitoring the change of the topology structure information, determine a plurality of newly queried slave databases according to the changed topology structure information;

Fourth connections are respectively established with the multiple slave databases obtained by the new query.
A configuration data access device is applied to a configuration management server of a configuration center in a microservice architecture, the configuration center includes a configuration center server and a configuration center client, and the configuration center server includes the configuration management server and a configuration server, the configuration management server is deployed in a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets the preset quality requirements, and the configuration server is deployed in at least two sets of second computer rooms. In the computer room, the same group of second computer rooms are located in the same business area, and different groups of second computer rooms are located in different business areas. Each first computer room is equipped with a master database, and each second computer room is equipped with a slave database. The slave database is included in the database cluster corresponding to the configuration center, and all master databases are set to ensure strong data consistency based on a preset consistency protocol;

The device includes:

Write event monitoring module, set to detect whether the configuration data write event is triggered;

The writing operation module is set to be triggered in response to detecting that the configuration data writing event is triggered, and in the case of detecting that the target main database corresponding to the first computer room is in a writable state, write configuration data to the target main database operations, wherein the master database in a writable state is determined based on the preset consistency protocol;

The write operation feedback module is configured to feed back write success information after detecting that the main databases in all the main databases except the target main database have completed data synchronization for this write operation.
A configuration data access device is applied to a configuration server of a configuration center in a microservice architecture, the configuration center includes a configuration center server and a configuration center client, the configuration center server includes a configuration management server and the configuration center server A configuration server, the configuration management server is deployed in a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets a preset quality requirement, and the configuration server is deployed in at least two sets of second computer rooms In the same group of second computer rooms are located in the same business area, different groups of second computer rooms are located in different business areas, each first computer room is equipped with a master database, each second computer room is equipped with a slave database, the master database and the The slave database is included in the database cluster corresponding to the configuration center, and all master databases are set to ensure strong data consistency based on a preset consistency protocol;

The device includes:

The read event triggers the detection module, which is set to detect whether the configuration data read event is triggered;

A state detection module, configured to detect an alternate slave database in a readable state in response to detecting that the configuration data read event is triggered;

The read operation module is set to select the purpose from the database from the alternative database, and perform a configuration data read operation from the database for the purpose, wherein the purpose is from the computer room where the database is located and the current configuration server is located. The first distance of the second computer room is smaller than the second distance between the computer room where the candidate secondary database except the destination secondary database is located in the candidate secondary database and the second computer room where the current configuration server is located.
A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the computer program described in claims 1-9 when the processor executes the computer program The configuration data access method of any one.
A configuration center in a microservice architecture includes a configuration center server and a configuration center client, the configuration center server includes a configuration management server and a configuration server, and the configuration management server is deployed in a plurality of first computer rooms The network quality between the multiple first computer rooms meets the preset quality requirements, the configuration server is deployed in at least two groups of second computer rooms, the same group of second computer rooms is located in the same business area, and different groups of second computer rooms are located in the same business area. The computer rooms are located in different business areas. Each first computer room is equipped with a master database, and each second computer room is equipped with a slave database. The master database and the slave database are included in the database cluster corresponding to the configuration center. All master databases Set to ensure strong data consistency based on the preset consistency protocol;

The configuration management server is set to perform configuration data access based on the configuration data access method according to any one of claims 1-5;

The configuration server is configured to perform configuration data access based on the configuration data access method according to any one of claims 6-9.
A computer-readable storage medium storing a computer program, wherein when the program is executed by a processor, the configuration data access method according to any one of claims 1-9 is implemented.