WO2022037359A1 - Procédé, appareil et dispositif d'accès à des données de configuration, centre de configuration et support de stockage - Google Patents

Procédé, appareil et dispositif d'accès à des données de configuration, centre de configuration et support de stockage Download PDF

Info

Publication number
WO2022037359A1
WO2022037359A1 PCT/CN2021/108147 CN2021108147W WO2022037359A1 WO 2022037359 A1 WO2022037359 A1 WO 2022037359A1 CN 2021108147 W CN2021108147 W CN 2021108147W WO 2022037359 A1 WO2022037359 A1 WO 2022037359A1
Authority
WO
WIPO (PCT)
Prior art keywords
configuration
database
computer
server
slave
Prior art date
Application number
PCT/CN2021/108147
Other languages
English (en)
Chinese (zh)
Inventor
邓辉超
Original Assignee
百果园技术(新加坡)有限公司
邓辉超
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202010831215.3A external-priority patent/CN112015744B/zh
Application filed by 百果园技术(新加坡)有限公司, 邓辉超 filed Critical 百果园技术(新加坡)有限公司
Publication of WO2022037359A1 publication Critical patent/WO2022037359A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2365—Ensuring data consistency and integrity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Definitions

  • the present application relates to the field of computer technology, for example, to a configuration data access method, apparatus, device, configuration center, and storage medium.
  • the configuration center has increasingly become one of the core basic components in the entire service governance architecture system.
  • Many other components in the service governance architecture system need to rely on the configuration center to achieve features such as automatic failover, high availability, and current limiting and fuse; while the business system needs to rely on the configuration center to achieve dynamic expansion and the ability to adjust behavior when the business system is running. Therefore, the configuration center needs to have high availability and effectively ensure data consistency.
  • the configuration center since the configuration center is the underlying component, it needs to minimize the dependence on other systems or components.
  • MMM Mysql Master-Master replication manager for Mysql
  • MHA Master High Availability
  • MGR MySQL Group Replication
  • MMM and MHA have the risk of data loss
  • MHA and MGA need to introduce other components or middleware to provide a unified entry, and the configuration center as the underlying component does not want to rely on other components
  • MMM The high availability of MHA and MGA is aimed at the master database, and is not suitable for direct use when the slave database is deployed in a large area.
  • the present application provides a configuration data access method, apparatus, device, configuration center and storage medium, which can optimize the configuration data access scheme of the configuration center in the microservice architecture.
  • the present application provides a configuration data access method, which is applied to a configuration management server of a configuration center in a micro-service architecture, where the configuration center includes a configuration center server and a configuration center client, and the configuration center server includes the A configuration management server and a configuration server, the configuration management server is deployed in a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets preset quality requirements, and the configuration server is deployed at least In two groups of second computer rooms, the same group of second computer rooms is located in the same business area, and different groups of second computer rooms are located in different business areas, each first computer room is equipped with a master database, and each second computer room is equipped with a slave database, the said The master database and the slave database are included in the database cluster corresponding to the configuration center, and all master databases are set to ensure strong data consistency based on a preset consistency protocol;
  • the method includes:
  • the configuration data writing operation is performed on the target master database, wherein the master database in the writable state is based on the preset consistency protocol Sure;
  • the writing success information is fed back.
  • the present application provides a configuration data access method, which is applied to a configuration server of a configuration center in a microservice architecture, where the configuration center includes a configuration center server and a configuration center client, and the configuration center server includes a configuration management service and the configuration server, the configuration management server is deployed in a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets the preset quality requirements, and the configuration server is deployed in at least two In a group of second computer rooms, the same group of second computer rooms are located in the same business area, and different groups of second computer rooms are located in different business areas.
  • Each first computer room is equipped with a master database
  • each second computer room is equipped with a slave database.
  • the database and the slave database are included in the database cluster corresponding to the configuration center, and all master databases are set to ensure strong data consistency based on a preset consistency protocol;
  • the method includes:
  • the purpose slave database is selected from the candidate slave database, and the configuration data read operation is performed from the database for the purpose, wherein the purpose is the first distance between the computer room where the database is located and the second computer room where the current configuration server is located. It is less than the second distance between the computer room where the candidate secondary database except the target secondary database is located in the candidate secondary database and the second computer room where the current configuration server is located.
  • the present application provides a configuration data access device, which is applied to a configuration management server of a configuration center in a microservice architecture.
  • the configuration center includes a configuration center server and a configuration center client, and the configuration center server includes the configuration center server.
  • a configuration management server and a configuration server the configuration management server is deployed in a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets preset quality requirements, and the configuration server is deployed at least In two groups of second computer rooms, the same group of second computer rooms is located in the same business area, and different groups of second computer rooms are located in different business areas, each first computer room is equipped with a master database, and each second computer room is equipped with a slave database, the said The master database and the slave database are included in the database cluster corresponding to the configuration center, and all master databases are set to ensure strong data consistency based on a preset consistency protocol;
  • the device includes:
  • Write event monitoring module set to detect whether the configuration data write event is triggered
  • the writing operation module is set to be triggered in response to detecting that the configuration data writing event is triggered, and in the case where it is detected that the target main database corresponding to the first computer room is in a writable state, write configuration data to the target main database operations, wherein the master database in a writable state is determined based on the preset consistency protocol;
  • the write operation feedback module is configured to feed back write success information after detecting that the master databases in all the master databases except the target master database have completed data synchronization for this write operation.
  • the present application provides a configuration data access device, which is applied to a configuration server of a configuration center in a microservice architecture, where the configuration center includes a configuration center server and a configuration center client, and the configuration center server includes a configuration management service and the configuration server, the configuration management server is deployed in a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets the preset quality requirements, and the configuration server is deployed in at least two In a group of second computer rooms, the same group of second computer rooms are located in the same business area, and different groups of second computer rooms are located in different business areas.
  • Each first computer room is equipped with a master database
  • each second computer room is equipped with a slave database.
  • the database and the slave database are included in the database cluster corresponding to the configuration center, and all master databases are set to ensure strong data consistency based on a preset consistency protocol;
  • the device includes:
  • the read event triggers the detection module, which is set to detect whether the configuration data read event is triggered;
  • a state detection module configured to detect an alternate slave database in a readable state in response to detecting that the configuration data read event is triggered
  • the read operation module is set to select the purpose from the database from the alternative database, and perform a configuration data read operation from the database for the purpose, wherein the purpose is from the computer room where the database is located and the current configuration server is located.
  • the first distance of the second computer room is smaller than the second distance between the computer room where the candidate secondary database except the destination secondary database is located in the candidate secondary database and the second computer room where the current configuration server is located.
  • the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the above-mentioned configuration data access method when the computer program is executed.
  • the application provides a configuration center in a micro-service architecture, including a configuration center server and a configuration center client, the configuration center server includes a configuration management server and a configuration server, and the configuration management server is deployed in multiple In a first computer room, the network quality between the multiple first computer rooms meets the preset quality requirements, the configuration server is deployed in at least two groups of second computer rooms, and the same group of second computer rooms is located in the same service area, Different groups of second computer rooms are located in different business areas, each first computer room is equipped with a master database, and each second computer room is equipped with a slave database, the master database and the slave database are included in the database cluster corresponding to the configuration center , all master databases are set to ensure strong data consistency based on preset consistency protocols;
  • the configuration management server is set to perform configuration data access based on the above-mentioned configuration data access method applied to the configuration management server;
  • the configuration server is configured to perform configuration data access based on the above-mentioned configuration data access method applied to the configuration server.
  • the present application provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned configuration data access method is implemented.
  • FIG. 1 is a scenario architecture diagram of an application scenario to which a configuration data access method provided by an embodiment of the present application is applicable;
  • FIG. 2 is a schematic diagram of a configuration center deployment solution provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of a database cluster deployment solution provided by an embodiment of the present application.
  • FIG. 4 is a schematic flowchart of a configuration data access method provided by an embodiment of the present application.
  • FIG. 5 is a schematic flowchart of another configuration data access method provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of an implementation manner of an intelligent data source provided by an embodiment of the present application.
  • FIG. 7 is a structural block diagram of a configuration data access device provided by an embodiment of the present application.
  • FIG. 8 is a structural block diagram of another configuration data access device provided by an embodiment of the present application.
  • FIG. 9 is a structural block diagram of a computer device according to an embodiment of the present application.
  • This embodiment mainly improves the configuration center in the microservice architecture.
  • the basic idea of microservices is to consider creating applications around business domain components that can be independently developed, managed and accelerated, using microservices cloud architecture and platforms in decentralized components, enabling deployment, management and service functions Interaction just got easier.
  • the configuration of the program is increasing, such as the switch of various functions, the configuration of parameters, and the address of the server, etc., and the expectations for configuration are also getting higher and higher, such as configuration After modification, it takes effect in real time, grayscale release, sub-environment or sub-cluster management configuration, perfect authority mechanism and audit mechanism, etc., adopt a distributed development model, so that the mutual reference between projects increases with the continuous increase of services.
  • the complexity of calling each other between projects increases exponentially, and the workload continues to increase each time a new project is put into production or online, so it is necessary to refer to the configuration center for governance.
  • FIG. 1 is a scenario architecture diagram of an application scenario to which a configuration data access method provided by an embodiment of the present application is applicable.
  • the application scenario may include a configuration center (Config Center) 100 , a database (DataBase, DB) 200 and a business system 300 .
  • the configuration center 100 includes a configuration center server 101 and a configuration center client (Config Client) 102, the configuration center server 101 includes a configuration management service (Config Admin Service) end 1011 and a configuration service (Config Service) end 1012, and the database 200 Including the master database 201 and the slave database 202.
  • the configuration center 100 can be understood as a complete set of systems that provide distributed configuration services.
  • the distributed configuration service can include functions such as configuration modification, publishing, and push.
  • the configuration management server 1011, the configuration server 1012, and the configuration center client 102 can be understood as Submodules in the configuration center 100 are configured.
  • the configuration server 1012 is generally a read-only server, providing configuration pull and push services for clients;
  • the configuration management server 1011 can provide configuration management services, including configuration viewing, modification, and publishing functions;
  • the configuration center client 102 can provide a software development kit (Software Development Kit, SDK) used by the business system, and interact with the configuration server 1012 to implement configuration pull and push functions.
  • the configuration center 100 needs to have high availability and effectively ensure data consistency. At the same time, since the configuration center 100 is a bottom-level component, it needs to minimize the dependence on other systems or components.
  • the configuration center 100 needs to manage the types and data volumes of the configuration data.
  • the configuration management server 1011 can perform write operations such as configuration modification for the master database 201.
  • the master database 201 and the slave database 202 perform data synchronization and configure services.
  • the terminal 1012 may perform a configuration read operation from the database 202 to provide configuration data to the configuration center client 102 .
  • the business system forms multiple business areas (for example, divided by continents, countries, regions or cities, etc.), and it is necessary to deploy equipment equipped with multiple sub-modules or databases of the configuration center in a large geographical scope.
  • equipment equipped with multiple sub-modules or databases of the configuration center in a large geographical scope.
  • the computer rooms where the servers are stored are widely distributed, and the communication distance between each two computer rooms is also greatly increased, so the network environment becomes more and more unreliable, and the reliability and availability of the configuration center implementation plan are more and more difficult to guarantee. .
  • MMM is a set of script programs that support dual-master failover and dual-master daily management. Since MMM cannot fully guarantee data consistency, MMM is only suitable for scenarios where data consistency requirements are not very high; MHA is MySQL A relatively mature solution in terms of high availability, the software consists of two parts, including a management node (MHA Manager) and a data node (MHA Node).
  • MHA Manager management node
  • MHA Node data node
  • MHA Manager will regularly detect the master (master) node in the cluster, when the master node When a node fails, it can automatically promote the slave (slave) node that stores the latest data to the new master node, and then redirect all other slave nodes to the new master node;
  • MGR is a replication plug-in based on MySQL architecture, using The Paxos protocol implements data consistency among multiple nodes, and MGR supports single-master mode and multi-master mode.
  • both MMM and MHA have the risk of data loss; MHA and MGA need to introduce other components (such as virtual IP address (Virtual IP Address, VIP) components) or database middleware to provide a unified entry, relying on additional operations.
  • VIP Virtual IP Address
  • the configuration center in the embodiment of the present application includes a configuration center server and a configuration center client, the configuration center server includes a configuration management server and a configuration server, and the configuration management server is deployed in a plurality of first computer rooms, The network quality between the plurality of first computer rooms meets the preset quality requirements, the configuration server is deployed in at least two groups of second computer rooms, the same group of second computer rooms is located in the same business area, and different groups of second computer rooms are located in the same business area.
  • each first computer room is equipped with a master database
  • each second computer room is equipped with a slave database.
  • the master database and the slave database are included in the database cluster corresponding to the configuration center. Set up a consistency protocol to ensure strong data consistency.
  • strong consistency can mean that any read operation can read the last written value of a data, or, at any time, the data in all nodes is the same, for example, for a relational database, it is required to update Subsequent access to the past data can be seen.
  • the number of the first computer room is not limited, that is, the number of deployments of the configuration management server is not limited, which can be set according to the actual situation (such as the number of business areas).
  • the number of the first computer room can be set as Three, one configuration management server can be integrated into computer equipment such as a server in the corresponding first computer room.
  • Each first computer room is equipped with a main database, that is, the number of the main databases and the number of the first computer rooms can be equal, that is, the number of the main databases can be equal to the deployment number of the configuration management server.
  • the configuration management server can access configuration data for its corresponding main database, and the device where the main database is located and the device where the configuration management server is located can be the same.
  • the types of the master database and the slave database are generally the same. This application does not limit the types of the master database and the slave database.
  • the types of the master database and the slave database can be relational databases such as MySQL, or can be a remote dictionary service (Remote dictionary server). , Redis) and other non-relational databases. All master databases ensure strong data consistency based on a preset consistency protocol. Therefore, the network quality between multiple first computer rooms needs to meet the preset quality requirements, and the network quality between multiple first computer rooms can meet the preset quality requirements.
  • the network quality between any two first computer rooms in all the first computer rooms meets the preset quality requirements, or there may be another first computer room corresponding to each first computer room in all the first computer rooms and each of the first computer rooms
  • the network quality between a computer room meets the preset quality requirements.
  • the network quality can be measured by indicators such as packet loss rate or round-trip time (RTT). Connection, to ensure that the network communication between each other is reliable.
  • the preset consistency protocol may be, for example, the Paxos protocol, or other protocols that can ensure strong data consistency between databases.
  • one or more master nodes can be elected in multiple master databases to be set for data writing, that is, it can be guaranteed that one or more master nodes can be set to write data, and other master databases automatically become
  • the slave node of the master node when writing configuration data to the master node, the master node synchronizes the written configuration data to the slave node, that is, the slave node can passively write the configuration data.
  • the preset consistency protocol will confirm the success of the write operation, thereby ensuring strong data consistency between the primary databases.
  • data synchronization between master-slave nodes or between dual-master nodes is asynchronous, that is to say, during a write operation, as long as one database completes the write, the next write can be performed. , if there is a problem with the database at this time, and other databases have not been synchronized in time, it will cause data loss.
  • strong data consistency is ensured between multiple master databases based on a preset consistency protocol, and data loss due to a problem in one of the master databases will not occur.
  • the slave database is deployed in the business area.
  • the division rules of business areas and the number of business areas can be determined according to the actual business scope and other related factors.
  • a group of second computer rooms can be deployed in each business area.
  • the number of second computer rooms in different groups can be the same or different.
  • the application does not limit the number of second computer rooms in each group of second computer rooms.
  • the number of second computer rooms is not limited, that is, the number of configuration servers deployed is not limited, and one configuration server can be integrated into computer equipment such as servers in the corresponding second computer room.
  • Each second computer room is equipped with a slave database
  • the number of slave databases can be equal to the number of second computer rooms, that is, the number of slave databases can be equal to the deployment quantity of the configuration server, and the configuration server can target its corresponding slave database.
  • the device where the database is located and the device where the configuration server is located can be the same.
  • the following is a schematic description of the configuration center deployment scheme and the database cluster deployment scheme. Take the master database and the slave database as MySQL databases, and the default consistency protocol as the Paxos protocol.
  • the first computer room corresponds to a business area, a group of second computer rooms is deployed in each business area, and the number of second computer rooms in each group of second computer rooms is three.
  • FIG. 2 is a schematic diagram of a configuration center deployment scheme provided by an embodiment of the present application.
  • the portal can be understood as a human-computer interaction interface operated by staff, and the staff can use the Portal to manage services (Configuration) according to the configuration.
  • the address information of the device where the Admin Service) end is located such as IP address and port number, to call the Config Admin Service to perform access operations such as writing configuration data, and the Config Admin Service end can write the configuration data to the corresponding master database.
  • three main databases can form an MGR group.
  • the MGR group realizes strong data consistency based on the Paxos protocol.
  • the Portal and Config Admin Service mainly provide configuration management, writing functions, and reading and writing the main database. Requires global deployment.
  • the Config Service side is set to implement configuration pull and real-time push for the business, which requires high availability and requires global deployment.
  • the Config Service side can select the nearest slave database to read configuration data, and each business area is set with a corresponding slave database. , such as three Asian slave databases, three European slave databases, and three Americas slave databases.
  • FIG 3 is a schematic diagram of a database cluster deployment solution provided by an embodiment of the present application.
  • an MGR Group consists of three nodes (Nodes), that is, three main databases, and the three nodes are elected through the Paxos protocol
  • a Master node is set to write data, and the data of the three nodes is kept strongly consistent through the Paxos protocol.
  • the configuration management server can perceive the role changes of the three nodes of the MGR Group and automatically select the Master node for configuration writing.
  • the three nodes are deployed in three computer rooms in the same city.
  • Slave databases are deployed on three continents around the world, and three computer rooms (second computer rooms) are deployed on each continent.
  • Asynchronous replication (Replicate) can be used to achieve eventual consistency between the slave database and the master database.
  • the configuration server of each second computer room preferentially reads the slave database in the computer room, and downgrades to read other slave databases in the same area when the slave database in the computer room is unavailable, so as to realize the mutual backup of the slave databases, so as to achieve high availability.
  • the implementation details can be See related content below.
  • the type and quantity of the master database the type and quantity of the slave database, the type of the preset consistency protocol, the division method of the business area, the number of the first computer room, and the second computer room in each group of the second computer room.
  • the number of computer rooms, etc. can be set according to actual needs.
  • FIG. 4 is a schematic flowchart of a configuration data access method provided by an embodiment of the present application.
  • the method is applied to a configuration management server of a configuration center in a microservice architecture, and can be executed by a configuration data access device, which can be configured by software and/or Or hardware implementation, generally can be integrated in computer equipment such as servers.
  • the method includes:
  • Step 401 it is detected that a configuration data write event is triggered.
  • the configuration data writing event here may be a configuration data writing event for the master database, and the triggering condition of the event may be set according to actual requirements. For example, when a staff member needs to perform configuration addition, configuration modification, configuration deletion or other configuration data change operations, the above-mentioned addition, modification and deletion can be regarded as write operations.
  • the configuration data writing to the main database is realized by calling the configuration management server.
  • the configuration management server receives the configuration data writing request sent by the configuration management device, it can be considered that the configuration data writing has been detected. event is fired.
  • This application does not limit the number of configuration management devices, which can be one or more. When the number is more than one, it can be the same as the number of the first computer room.
  • the configuration management devices can be set in the first computer room or in Other locations can communicate with the device where the configuration management server is located.
  • Step 402 If it is detected that the target master database corresponding to the first computer room is in a writable state, perform a configuration data writing operation for the target master database, wherein the master database in the writable state is based on the preset consistency. Agreement is confirmed.
  • multiple master databases may elect a master database as a master node based on a preset consistency protocol, and the master database as a master node is in a writable state.
  • the number of master nodes may be one or more, and this application does not limit the number of master nodes. If the target master database corresponding to the first computer room where the current configuration management server is located is the master node, the target master database is in a writable state, and the current configuration management server can detect that the target master database is writable, and then can target the target master database.
  • the primary database performs configuration data write operations. If the target master database corresponding to the first computer room where the current configuration management server is located is not the master node, other configuration management servers will detect the writable master database, thereby realizing highly reliable configuration data writing.
  • Step 403 After detecting that the primary database other than the target primary database has completed the data synchronization for this writing operation, feedback the writing success information.
  • the target master database in the process of writing configuration data to the target master database, since all master databases ensure strong data consistency based on a preset consistency protocol, the target master database will synchronize the written configuration data to other master databases. In the database, after detecting that other primary databases have completed data synchronization for this write operation, it can confirm that the configuration data to be written has been successfully written to all primary databases, and can feedback the successful writing information to the configuration management device. for subsequent operations.
  • the configuration center in the micro-service architecture includes a configuration center server and a configuration center client
  • the configuration center server includes a configuration management server and a configuration server
  • the configuration management server is deployed in In multiple first computer rooms, each first computer room is equipped with a master database. All master databases ensure strong data consistency based on a preset consistency protocol.
  • the network quality between the first computer rooms meets the preset quality requirements, which can ensure that the first computer room is At the same time, it can also ensure the synchronization performance between the main databases and the high availability of the main database.
  • the configuration server is deployed in at least two sets of second computer rooms, and the same group of second computer rooms are located in the same business area.
  • each second computer room is equipped with a slave database.
  • the master database and the slave database are included in the database cluster corresponding to the configuration center.
  • the configuration management server Based on the architecture of the configuration center, the configuration management server detects that the After the configuration data write event is triggered, it actively detects whether the target master database corresponding to the first computer room is in a writable state. If the target master database corresponding to the first computer room is in a writable state, the configuration data is written to the target master database.
  • the main database in the writable state is determined based on the preset consistency protocol, that is, the master node defined in the preset consistency protocol, so as to ensure strong data consistency between the master databases based on this protocol, and detect After the data synchronization for this write operation is completed in the main database other than the target main database, the write success information is fed back, so that the intelligent detection of the writable main database and the configuration data can be realized in the configuration center without relying on other components.
  • High-reliability write operations, and all primary databases are based on preset consistency protocols to ensure high availability and data consistency of the primary databases, avoiding business impact due to data loss.
  • the method before the detecting that the configuration data writing event is triggered, the method further includes: acquiring topology structure information of the database cluster, wherein the topology structure information includes the corresponding relationship between the computer room and the database; The topological structure information is queried in the target master database corresponding to the first computer room where it is located.
  • the topology information can be included in a database topology configuration file, and the application does not limit the format of the configuration file, for example, it can be in a JS Object Notation (JavaScript Object Notation, JSON) format, such as a database topology configuration file. Can be recorded as datasource.json.
  • the topology information can include a list of master databases and a list of slave databases.
  • the master database list may include the IP and port number of each master database
  • the slave database list may include computer room information and location information (such as local area information) corresponding to each slave database.
  • the configuration management server can confirm the IP and port number of its own device, and then compare its own IP and port number with the IP and port number in the main database list, and find the highest similarity (if the device where the configuration management server is located and If the device where the corresponding master database is located is the same, then the master database with the same IP and port number is determined as its corresponding target master database.
  • the method further includes: establishing a first connection with the target master database; detecting the target master database Being in a writable state includes: attempting to write first data to the target primary database based on the first connection, and determining that the target primary database is in a writable state if it is detected that the writing of the first data is successful.
  • the method further includes: if a change in the topology structure information is monitored, determining a new target master database according to the changed topology structure information; The new target master database establishes a second connection.
  • the advantage of this setting is that if the topology information changes, the corresponding target master database may also change, and the first connection with the original target master database can be disconnected, and the second connection with the new target master database can be established.
  • the method further includes: detecting that a configuration data read event is triggered; detecting a candidate master database in a readable state; selecting a target master database from the candidate master databases, and targeting the purpose
  • the main database performs a configuration data read operation, wherein the first distance between the computer room where the target main database is located and the first computer room where the current configuration management server is located is smaller than the first computer room where the other candidate master databases are located and the first computer room where the current configuration management server is located. the second distance.
  • the advantage of this setting is that the configuration management server may also need to read the configuration data. When the configuration management server needs to read the configuration data in the main database, it can select the nearest main database for the read operation.
  • the main database of the purpose is the main database of the same equipment or the same computer room as the current configuration management server.
  • the nearest main database can also be selected from the main databases in the readable state for reading. operate.
  • the detection method may be one by one according to the distance between the computer room where the main database is located and the first computer room where the current configuration management server is located, or may be batch detection. The application does not limit the detection method.
  • FIG. 5 is a schematic flowchart of another configuration data access method provided by an embodiment of the present application.
  • the method is applied to a configuration server of a configuration center in a microservice architecture, and can be executed by a configuration data access device, which can be implemented by software and/or Or hardware implementation, generally can be integrated in computer equipment such as servers.
  • the method includes:
  • Step 501 it is detected that a configuration data read event is triggered.
  • the configuration data reading event here may be an event for reading configuration data from a database, and the triggering condition of the event may be set according to actual requirements.
  • the business system when it needs to obtain configuration data, it can send a configuration data acquisition request to the configuration server through the configuration center client.
  • the configuration server needs to read the configuration data from the database.
  • the configuration server receives the request, It can be considered that the configuration data read event is triggered; for another example, the configuration server can push the configuration data to the business system according to the preset push rules (such as timing or real-time push), and when it is detected that the configuration data needs to be pushed, the configuration service
  • the client needs to read the configuration data from the database, and it can be considered that the configuration data read event is triggered.
  • a configuration server can correspond to one or more configuration center clients, that is, a configuration server can provide configuration pull or push services for one or more business systems.
  • Step 502 Detect a candidate slave database in a readable state.
  • the detection range may be a service area corresponding to the currently configured server, and detection operations may be reduced. To further ensure availability, the detection range may also be multiple service areas.
  • the configuration server in the embodiment of the present application can automatically detect the candidate slave database in a readable state, so as to facilitate the subsequent selection of the optimal slave database to perform the read operation of configuration data.
  • the detection method may be one by one according to the distance between the computer room where the database is located and the computer room where the current configuration server is located, or may be batch detection, which is not limited in this application.
  • Step 503 select the purpose from the database from the alternative database, and perform a configuration data read operation from the database for the purpose, wherein the purpose is from the computer room where the database is located and the second computer room where the current configuration server is located.
  • the first distance is smaller than the second distance between the computer room where the other candidate slave databases are located and the second computer room where the current configuration server is located.
  • the closest slave database may be selected from the candidate slave databases in the readable state to read the configuration data.
  • the slave database that is in the same device or room as the configuration server is the closest. If the slave database is abnormal and is not in a readable state, other slave databases in the same business area can be selected to read the configuration data.
  • the granularity of the distance can be freely set, which can be the distance of the actual geographic coordinates, or it can be a city or a region. For example, if the granularity of the distance is a city, the distance between the computer rooms in the same city can be considered as 0, and the distance between the computer rooms in two adjacent cities can be considered as 1 and so on.
  • the configuration center in the micro-service architecture includes a configuration center server and a configuration center client
  • the configuration center server includes a configuration management server and a configuration server
  • the configuration management server is deployed in In multiple first computer rooms, each first computer room is equipped with a master database. All master databases ensure strong data consistency based on a preset consistency protocol.
  • the network quality between the first computer rooms meets the preset quality requirements, which can ensure that the first computer room is At the same time, it can also ensure the synchronization performance between the main databases and the high availability of the main database.
  • the configuration server is deployed in at least two sets of second computer rooms, and the same group of second computer rooms are located in the same business area. Different groups of second computer rooms are located in different business areas.
  • Each second computer room is equipped with a slave database.
  • the master database and the slave database are included in the database cluster corresponding to the configuration center.
  • the configuration server Based on the architecture of the configuration center, the configuration server detects the configuration After the data read event is triggered, it actively detects the candidate slave database in the readable state, and selects the candidate slave database closest to the current configuration server to read the configuration data, realizes the mutual backup of the slave databases, and ensures the configuration center. high availability.
  • the method before the detecting that the configuration data read event is triggered, the method further includes: acquiring topology structure information of the database cluster, wherein the topology structure information includes information about the correspondence between the computer room and the database, And the corresponding relationship information between the database and the business area; according to the topology information, query the multiple slave databases contained in the target business area corresponding to the second computer room;
  • the detection of the candidate slave database in a readable state includes: A candidate slave database in a readable state is detected from the queried multiple slave databases.
  • the topology information can be included in the database topology configuration file, and the topology information in the configuration server can be the same as the topology information in the configuration management server, that is, the configuration server and the configuration management server belong to The same database topology configuration file may exist in all devices, which facilitates the unified writing, unified update and unified distribution of the database topology configuration file.
  • the topology information in the configuration server can also only include the slave database list. The configuration server can confirm the region and computer room to which it belongs, and then query the slave database that is the same as its own region and computer room in the slave database list.
  • the method further includes: separately establishing a third slave database with the queried multiple slave databases. connecting; the detecting a candidate slave database in a readable state from the queried multiple slave databases includes: trying to read the second data from the multiple queried slave databases based on the third connection. , if it is detected that the second data is successfully read, it is determined that the corresponding slave database is in a readable state, and the determined slave database is used as an alternate slave database.
  • the state of the slave database can be accurately judged.
  • the third connection with the slave database in the same device or in the same computer room as the current configuration server can be established first.
  • a read operation if the slave database is detected based on the third connection
  • a third connection is established with other slave databases, and an alternate slave database in a readable state is detected.
  • the method further includes: if it is monitored that the topology information changes, determining a new query query according to the changed topology information.
  • the multiple slave databases obtained by the new query respectively establish a fourth connection with the multiple slave databases obtained by the new query.
  • the configuration data access method applied to the configuration management server and the configuration data access method applied to the configuration server can be completed by the same program code, and the program can intelligently determine that the configuration data write is currently triggered.
  • the event is still a configuration data read event. Whether the configuration data read event is for the master database or the slave database can be judged according to the received request type or Structured Query Language (SQL) statement.
  • SQL Structured Query Language
  • FIG. 6 is a schematic diagram of an implementation manner of an intelligent data source provided in an embodiment of the present application.
  • datasource.json is the above-mentioned Database topology configuration file.
  • DataSourceConfigSyncer configures a listener for the data source, which is responsible for monitoring changes in datasource.json and notifying the DataSourceManager to rebuild the smart data source.
  • SmartDataSource is the implementation class of smart data source, which provides a unified data access interface to the upper layer. It contains the write data source MultiWriteDataSource and the read data source MultiReadDataSource, and route to the corresponding data source according to the type of SQL.
  • MultiWriteDataSource is the encapsulation of write data source, which contains multiple (for example, 3, the same as the number of main database) WriteDataSource, MultiWriteDataSource will select a writable WriteDataSource to write, and whether it is writable is determined by write heartbeat (WriteHeartbeat)
  • the write SQL is periodically sent for detection.
  • the single-master mode in the MGR may be adopted. In this mode, only one node can write at any time.
  • MultiReadDataSource is the encapsulation of read data source, which contains multiple (for example, a business area deploys 3 slave databases, the number here can be 3) ReadDataSource, MultiReadDataSource will read from all healthy (that is, in readable state) ReadDataSource Select an optimal ReadDataSource for read operations.
  • the health status is detected by periodically sending read SQL through the read heartbeat (ReadHeartbeat), and the selection of the optimal ReadDataSource is based on the priority field indication (select the smallest priority value).
  • the priority is sorted by distance, that is, The smaller the distance, the smaller the corresponding priority value.
  • FIG. 7 is a structural block diagram of a configuration data access device provided by an embodiment of the present application.
  • the device is applied to a configuration management server of a configuration center in a microservice architecture, and can be implemented by software and/or hardware, and can generally be integrated in a server, etc.
  • configuration data access may be performed by executing a configuration data access method.
  • the device includes:
  • the write event monitoring module 701 is set to detect whether the configuration data write event is triggered; the write operation module 702 is set to, after detecting that the configuration data write event is triggered, if the target host corresponding to the first computer room where it is located is detected If the database is in a writable state, the configuration data writing operation is performed on the target master database, wherein the master database in the writable state is determined based on the preset consistency protocol; the write operation feedback module 703 is set to detect After the master database other than the target master database completes the data synchronization for this writing operation, the writing success information is fed back.
  • the configuration center in the micro-service architecture includes a configuration center server and a configuration center client
  • the configuration center server includes a configuration management server and a configuration server
  • the configuration management server is deployed in multiple
  • each first computer room is equipped with a master database. All the master databases ensure strong data consistency based on the preset consistency protocol.
  • the network quality between the first computer rooms meets the preset quality requirements, which can ensure the first computer room. At the same time, it can also ensure the synchronization performance between the main databases and the high availability of the main database.
  • the configuration server is deployed in at least two groups of second computer rooms.
  • the same group of second computer rooms are located in the same business area, but different The second computer room of the group is located in different business areas.
  • Each second computer room is equipped with a slave database.
  • the master database and the slave database are included in the database cluster corresponding to the configuration center.
  • the configuration management server Based on the architecture of the configuration center, the configuration management server detects the configuration After the data write event is triggered, it actively detects whether the target master database corresponding to the first computer room is in a writable state. If the target master database corresponding to the first computer room is in a writable state, the configuration data is written to the target master database.
  • the master database in the writable state is determined based on the preset consistency protocol, that is, the master node defined in the preset consistency protocol, so as to ensure strong data consistency between master databases based on this protocol.
  • the write success information is fed back, so that the intelligent detection of the writable master database and the high level of configuration data can be realized in the configuration center without relying on other components.
  • Reliable write operations, and all primary databases ensure high availability and data consistency of the primary databases based on preset consistency protocols, avoiding business impact due to data loss.
  • FIG. 8 is a structural block diagram of another configuration data access device provided by an embodiment of the present application.
  • the device is applied to a configuration server of a configuration center in a microservice architecture, and can be implemented by software and/or hardware, and can generally be integrated in a server, etc.
  • configuration data access may be performed by executing a configuration data access method.
  • the device includes:
  • the read event trigger detection module 801 is configured to detect whether the configuration data read event is triggered; the state detection module 802 is configured to detect an alternative slave database in a readable state after detecting that the configuration data read event is triggered
  • the read operation module 803 is set to select the purpose from the database from the alternative database, and carry out the configuration data reading operation from the database for the purpose, wherein, the purpose is from the computer room where the database is located and the current configuration service.
  • the first distance of the second computer room where the terminal is located is smaller than the second distance between the computer room where the other candidate slave databases are located and the second computer room where the current configuration server is located.
  • the configuration center in the micro-service architecture includes a configuration center server and a configuration center client
  • the configuration center server includes a configuration management server and a configuration server
  • the configuration management server is deployed in In multiple first computer rooms, each first computer room is equipped with a master database. All master databases ensure strong data consistency based on a preset consistency protocol.
  • the network quality between the first computer rooms meets the preset quality requirements, which can ensure that the first computer room is At the same time, it can also ensure the synchronization performance between the main databases and the high availability of the main database.
  • the configuration server is deployed in at least two sets of second computer rooms, and the same group of second computer rooms are located in the same business area. Different groups of second computer rooms are located in different business areas.
  • Each second computer room is equipped with a slave database.
  • the master database and the slave database are included in the database cluster corresponding to the configuration center.
  • the configuration server Based on the architecture of the configuration center, the configuration server detects the configuration After the data read event is triggered, it actively detects the candidate slave database in the readable state, and selects the candidate slave database closest to the current configuration server to read the configuration data, realizes the mutual backup of the slave databases, and ensures the configuration center. high availability.
  • FIG. 9 is a structural block diagram of a computer device according to an embodiment of the present application.
  • the computer device 900 includes a memory 901, a processor 902, and a computer program that is stored in the memory 901 and can run on the processor 902.
  • the processor 902 executes the computer program, the configuration data access method provided by the embodiments of the present application is implemented. .
  • the embodiments of the present application further provide a storage medium containing computer-executable instructions, where the computer-executable instructions, when executed by a computer processor, are used to execute the configuration data access method provided by the embodiments of the present application.
  • Embodiments of the present application further provide a configuration center in a micro-service architecture, including a configuration center server and a configuration center client, the configuration center server includes a configuration management server and a configuration server, and the configuration management server deploys In a plurality of first computer rooms, the network quality between the plurality of first computer rooms meets the preset quality requirements, the configuration server is deployed in at least two groups of second computer rooms, and the same group of second computer rooms is located in the same service different groups of second computer rooms are located in different business areas, each first computer room is equipped with a master database, and each second computer room is equipped with a slave database, the master database and the slave database are included in the database corresponding to the configuration center In the cluster, all main databases ensure strong data consistency based on a preset consistency protocol; the configuration management server performs access operations such as configuration data writing based on the configuration data access method described in the embodiment of the present application; the configuration server is based on the The configuration data access method described in the embodiments of the present application performs access operations such as configuration data reading.
  • the configuration data access device, device, storage medium, and configuration center provided in the above embodiments can execute the configuration data access method provided by the embodiment of the present application, and have corresponding functional modules and effects for executing the method.
  • the configuration data access method provided by the corresponding embodiment of the present application can execute the configuration data access method provided by the corresponding embodiment of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Procédé, appareil et dispositif d'accès à des données de configuration, centre de configuration et support de stockage. Le procédé d'accès aux données de configuration est appliqué à un serveur de gestion de configuration. Le serveur de gestion de configuration est déployé dans de multiples premières salles des ordinateurs et un serveur de configuration est déployé dans au moins deux ensembles de secondes salles des ordinateurs ; chaque première salle des ordinateurs est équipée d'une base de données principale, chaque seconde salle des ordinateurs est équipée d'une base de données secondaire et la base de données principale est configurée pour garantir une forte cohérence des données sur la base d'un protocole de cohérence prédéfini. Le procédé consiste : à détecter qu'un événement d'écriture de données de configuration est déclenché (401) ; lorsqu'il est détecté qu'une base de données principale cible, qui correspond à la première salle des ordinateurs dans laquelle elle est située, est dans un état inscriptible, à effectuer une opération d'écriture de données de configuration pour la base de données principale cible, la base de données principale dans l'état inscriptible étant déterminée sur la base d'un protocole de cohérence prédéfini (402) ; et une fois que toutes les bases de données principales à l'exception de la base de données principale cible ont achevé une synchronisation de données pour la présente opération d'écriture, à envoyer en rétroaction des informations de réussite d'écriture (403).
PCT/CN2021/108147 2020-08-18 2021-07-23 Procédé, appareil et dispositif d'accès à des données de configuration, centre de configuration et support de stockage WO2022037359A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010831215.3 2020-08-18
CN202010831215.3A CN112015744B (zh) 2020-08-18 配置数据访问方法、装置、设备、配置中心及存储介质

Publications (1)

Publication Number Publication Date
WO2022037359A1 true WO2022037359A1 (fr) 2022-02-24

Family

ID=73504895

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/108147 WO2022037359A1 (fr) 2020-08-18 2021-07-23 Procédé, appareil et dispositif d'accès à des données de configuration, centre de configuration et support de stockage

Country Status (1)

Country Link
WO (1) WO2022037359A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114844951A (zh) * 2022-04-22 2022-08-02 百果园技术(新加坡)有限公司 请求处理方法、系统、设备、存储介质及产品
CN115934428A (zh) * 2023-01-10 2023-04-07 湖南三湘银行股份有限公司 一种mysql数据库的主灾备切换方法、装置及电子设备

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105205182A (zh) * 2015-10-28 2015-12-30 北京奇虎科技有限公司 多机房部署系统及跨机房的业务数据处理方法
CN106341454A (zh) * 2016-08-23 2017-01-18 世纪龙信息网络有限责任公司 跨机房多活分布式数据库管理系统和方法
CN108090222A (zh) * 2018-01-05 2018-05-29 中国科学院计算技术研究所 一种数据库集群节点间数据同步系统
CN109471755A (zh) * 2018-11-14 2019-03-15 江苏鸿信系统集成有限公司 一种基于云计算的同城异地灾备的方法及其系统
CN109726250A (zh) * 2018-12-27 2019-05-07 星环信息科技（上海）有限公司 数据存储系统、元数据库同步及数据跨域计算方法
US20190250575A1 (en) * 2018-02-13 2019-08-15 Tridonic Gmbh & Co Kg Commissioning smart lighting systems
CN110633175A (zh) * 2019-09-27 2019-12-31 掌阅科技股份有限公司 基于微服务的多机房数据处理方法、电子设备及存储介质
CN110851528A (zh) * 2019-10-09 2020-02-28 网联清算有限公司 数据库同步方法、装置、存储介质及计算机设备
CN111078426A (zh) * 2019-12-03 2020-04-28 紫光云(南京)数字技术有限公司 一种后端微服务架构下的高并发解决方法
CN111090637A (zh) * 2019-12-20 2020-05-01 辽宁振兴银行股份有限公司 一种金融级基于A10的高可用MySQL数据库实现方法
CN112015744A (zh) * 2020-08-18 2020-12-01 广州市百果园信息技术有限公司 配置数据访问方法、装置、设备、配置中心及存储介质

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105205182A (zh) * 2015-10-28 2015-12-30 北京奇虎科技有限公司 多机房部署系统及跨机房的业务数据处理方法
CN106341454A (zh) * 2016-08-23 2017-01-18 世纪龙信息网络有限责任公司 跨机房多活分布式数据库管理系统和方法
CN108090222A (zh) * 2018-01-05 2018-05-29 中国科学院计算技术研究所 一种数据库集群节点间数据同步系统
US20190250575A1 (en) * 2018-02-13 2019-08-15 Tridonic Gmbh & Co Kg Commissioning smart lighting systems
CN109471755A (zh) * 2018-11-14 2019-03-15 江苏鸿信系统集成有限公司 一种基于云计算的同城异地灾备的方法及其系统
CN109726250A (zh) * 2018-12-27 2019-05-07 星环信息科技（上海）有限公司 数据存储系统、元数据库同步及数据跨域计算方法
CN110633175A (zh) * 2019-09-27 2019-12-31 掌阅科技股份有限公司 基于微服务的多机房数据处理方法、电子设备及存储介质
CN110851528A (zh) * 2019-10-09 2020-02-28 网联清算有限公司 数据库同步方法、装置、存储介质及计算机设备
CN111078426A (zh) * 2019-12-03 2020-04-28 紫光云(南京)数字技术有限公司 一种后端微服务架构下的高并发解决方法
CN111090637A (zh) * 2019-12-20 2020-05-01 辽宁振兴银行股份有限公司 一种金融级基于A10的高可用MySQL数据库实现方法
CN112015744A (zh) * 2020-08-18 2020-12-01 广州市百果园信息技术有限公司 配置数据访问方法、装置、设备、配置中心及存储介质

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114844951A (zh) * 2022-04-22 2022-08-02 百果园技术(新加坡)有限公司 请求处理方法、系统、设备、存储介质及产品
CN114844951B (zh) * 2022-04-22 2024-03-19 百果园技术(新加坡)有限公司 请求处理方法、系统、设备、存储介质及产品
CN115934428A (zh) * 2023-01-10 2023-04-07 湖南三湘银行股份有限公司 一种mysql数据库的主灾备切换方法、装置及电子设备

Also Published As

Publication number Publication date
CN112015744A (zh) 2020-12-01

Similar Documents

Publication Publication Date Title
US11360854B2 (en) Storage cluster configuration change method, storage cluster, and computer system
US10713134B2 (en) Distributed storage and replication system and method
CN108632067B (zh) 容灾部署方法、装置及系统
CN109729111B (zh) 用于管理分布式系统的方法、设备和计算机程序产品
CN107391294B (zh) 一种ipsan容灾系统的建立方法及装置
WO2022037359A1 (fr) Procédé, appareil et dispositif d'accès à des données de configuration, centre de configuration et support de stockage
CN105471622B (zh) 一种基于Galera的控制节点主备切换的高可用方法及系统
CN108234302A (zh) 保持网络装置用的分布式操作系统中的一致性
US11095742B2 (en) Query proxy for delivery of dynamic system state
WO2023082800A1 (fr) Procédé de sélection de nœud principal, base de données répartie et support de stockage
CN108897641B (zh) 一种数据库主备环境下的日志分析服务实时同步系统
CN103986789A (zh) 一种实现基于nfs的hadoop ha集群中nfs节点双机冗余的方法
Suh et al. On performance of OpenDaylight clustering
CN115794499B (zh) 一种用于分布式块存储集群间双活复制数据的方法和系统
CN109639773A (zh) 一种动态构建的分布式数据集群控制系统及其方法
CN114116912A (zh) 一种基于Keepalived实现数据库高可用的方法
WO2017071384A1 (fr) Procédé et appareil de traitement de message
CN112000444B (zh) 数据库事务处理方法、装置、存储介质和电子设备
WO2015196692A1 (fr) Système informatique en nuage, et procédé et appareil de traitement pour un système informatique en nuage
CN103780433B (zh) 自愈式虚拟资源配置管理数据架构
CN116055314A (zh) 一种配置同步方法及装置
CN114172821B (zh) 服务状态的同步方法、装置及服务器
CN114500289B (zh) 控制平面恢复方法、装置、控制节点及存储介质
CN112015744B (zh) 配置数据访问方法、装置、设备、配置中心及存储介质
CN113472891A (zh) 一种sdn控制器集群数据处理方法、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21857454

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21857454

Country of ref document: EP

Kind code of ref document: A1