CN113032431A - High-availability client load balancing method based on database middleware cluster - Google Patents
High-availability client load balancing method based on database middleware cluster Download PDFInfo
- Publication number
- CN113032431A CN113032431A CN202110439382.8A CN202110439382A CN113032431A CN 113032431 A CN113032431 A CN 113032431A CN 202110439382 A CN202110439382 A CN 202110439382A CN 113032431 A CN113032431 A CN 113032431A
- Authority
- CN
- China
- Prior art keywords
- middleware
- connection
- client
- data source
- instance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24532—Query optimisation of parallel queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/217—Database tuning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
Abstract
The invention discloses a high-availability client load balancing method based on a database middleware cluster, which is characterized by comprising the following steps of 1: loading the client side to obtain a first connection credential of the client side and the database middleware cluster; step 2: generating a data source connection object by using the first connection certificate and storing the data source connection object into the client; and step 3: acquiring all online middleware examples from a database middleware cluster; and 4, step 4: after the first connection in the step 3 is successful, periodically checking the middleware instance state in the database middleware cluster; and 5: and the service system acquires the connection middleware instance according to the configured load balancing algorithm. The effects of realizing data communication based on the JDBC protocol, automatically processing the problems of connection, load, abnormity, fusing and the like between the data base middleware clusters and reducing the trouble of configuring load balancing hardware and software at a server are achieved.
Description
Technical Field
The invention belongs to the field of database middleware, and particularly relates to a high-availability client load balancing method based on a database middleware cluster.
Background
With the rapid development of the internet industry and the continuous increase of the business volume, a large amount of data is generated and recorded in a database every day. In the past, the data of a single table of the database is too much, and the reading performance is reduced. Meanwhile, single-point deployment of the traditional relational database cannot meet the requirements of enterprises on high performance and high availability of applications. Therefore, more and more businesses have begun to transform the use of distributed databases in recent years. However, complex distributed database environments place higher demands on the application developers and make troubleshooting more difficult. In order to solve the above problems, a transparent distributed database to applications is realized, and a database middleware system is developed to uniformly manage the distributed database and process all requests of the applications to the database. As a database middleware system, in order to ensure high performance and high availability of the database middleware system, the system adopts multi-instance cluster deployment, that is, a plurality of application instances are deployed in the database middleware system; the application instance is a data management program located between an application and an operating system, and any operation of any data in a database by the application, including data definition, data query, data maintenance, database operation control and the like, is completed through the application instance.
In order to ensure that each instance in the database middleware cluster can share a certain flow and automatically go off-line when a fault occurs, a reliable load balancing scheme is urgently needed. The traditional load balancing scheme is generally deployed at a server side, namely a middleware side of a database, and mainly comprises a hardware load balancing scheme and a software load balancing scheme, wherein the hardware load balancing scheme mainly comprises equipment specially used for load balancing which is installed among server nodes, and the software load balancing scheme is to complete request distribution work by installing some software used for load balancing functions or modules and the like on a server, such as Nginx and the like. However, deploying server load balancing hardware or software often requires professional operation and maintenance personnel to maintain, and meanwhile, any one of the database instances in the database middleware needs to be notified to the operation and maintenance personnel to modify configuration online and offline, so that an application using the database middleware cannot adjust a database access policy according to actual business needs of the application.
In addition to the server side configuration load balancing, client side configuration load balancing also appears in the market at present, mainly represented by Ribbon developed by Netflix corporation, which is called Spring Cloud Ribbon, and uses TCP or HTTP protocol for communication, and the client side configuration load balancing tool is a client side load balancing tool based on HTTP and TCP, but cannot meet the requirements of database middleware software based on JDBC protocol.
CN202010334190 provides a method for a client to connect a database based on JDBC protocol, and obtains a database connection address through a distributed data access layer middleware, thereby reducing the number of connections on a physical database, but the distributed database access layer middleware only solves the problem of fast connecting the database, and does not consider the problem of data acquisition in network congestion and connection failure environments; CN201510195238 provides a method for obtaining database cluster state notification information through a client, data operation requests of service applications are adaptively distributed to active database instances according to database state changes, however, in a distributed database environment, all services such as service applications have higher development requirements for developers through connection of the client and the database cluster due to processing of relationships of multiple databases, and availability and concurrency performance cannot be guaranteed.
Disclosure of Invention
The invention aims to enable a business system to conveniently, quickly, uniformly, efficiently and safely call the database middleware system through the client provided by the database middleware system. The database middleware client can automatically process the problems of network connection, communication, load, abnormity and the like among a plurality of instances. Application developers no longer need to be concerned with non-business problems between the processing and database middleware systems, so that the centers of work can be put on the businesses of the developers.
The technical scheme of the invention is as follows: a high-availability client load balancing scheme for a database middleware cluster specifically comprises the following steps:
step 1: the loading of the client side obtains the first connection credential of the client side and the database middleware cluster, and specifically comprises the following steps: compiling a JAR package of a client in Java language, wherein the client is a client of the database middleware, and importing the JAR package of the client into an application; in the process that a JAR packet of a client executes registration driving in an application, a middleware instance is arbitrarily selected from a database middleware cluster, and information of the middleware instance is configured as an initial connection parameter, wherein the information is used as a first connection evidence of the client and the database middleware cluster; the initial connection parameters include: middleware instance URL, account, password, IP and port number, maximum connection number, minimum connection number; the maximum connection number is used for limiting a client to establish connection with a maximum number of middleware instances; the minimum connection number is used for limiting how many intermediate instances a client at least keeps establishing connection with; if the number of the current client side connection middleware instances exceeds the preset maximum connection number, the connection request initiated by the client side needs to wait; the state of the middleware instance comprises an online state and a offline state; the middleware instance is on line, the availability of the middleware instance is represented, and the client can be connected with the database middleware cluster through the instance; the offline of the middleware instance represents that the middleware instance is unavailable, and the client cannot be connected with the database middleware cluster through the instance;
step 2: the method comprises the following steps of generating a data source connection object by using a first connection certificate and storing the data source connection object into a client, specifically: creating an available data source set and a data source set to be offline in the client, extracting the IP and the port number of the middleware instance according to the initial connection parameters obtained in the step 1, generating a default data source connection object and storing the default data source connection object in the available data source set; the data source connection object and the middleware instance keep a one-to-one correspondence relationship; the available data source set is used for storing the middleware instances in an online state, which are obtained from the database middleware cluster; the data source set to be offline is used for storing the middleware instances in the offline state in the database middleware cluster;
and step 3: acquiring all online middleware instances from a database middleware cluster: the client acquires a default data source connection object from the available data source set, determines a pre-connected middleware instance according to the IP and the port number in the data source connection object, extracts a URL (Uniform resource locator) address in initial connection parameters, accesses a database middleware cluster by a JDBC (joint discovery code) protocol and is connected with the middleware instance corresponding to the data source connection object; simultaneously requesting to acquire all online middleware instances and instance information in a current database middleware cluster, generating connection parameters of the middleware instances, creating a data source connection object corresponding to the instances at a client, and putting the data source connection object into an available data source set of the client;
and 4, step 4: after the first connection in the step 3 is successful, periodically checking the middleware instance state in the database middleware cluster, and reallocating the data source connection objects in the available data source set and the data source connection objects in the data source set to be offline of the client according to the middleware instance state;
and 5: the service system obtains a connection middleware example according to a configured load balancing algorithm: the service system generates a database access request, triggers a database middleware client in the service system to select a data source connection object according to a preset load balancing algorithm, analyzes connection parameters and requests to access a middleware instance of a corresponding data source connection object in a database middleware cluster according to the connection parameters; before the client establishes connection with the middleware instance, the connection of the client is confirmed to not reach the maximum connection number, and if the connection is reached, the connection request needs to wait for post-processing.
In the step 1, in the connection process between the client and the middleware instance, the connection condition between the middleware client and the middleware instance is monitored in real time, and the number of the middleware instances which are successfully connected with the client at present is counted according to the number of the client connections; when the client is successfully connected with the middleware instance through the data source connection object, adding 1 to the client connection number; when the client's connection to the middleware instance ends, the client connection number is decremented by "1".
In step 1, the loading method of the client includes: for the Java project, loading the JAR package of the client side by putting the JAR package into a specified lib directory, and for the MAVEN project, writing a loading code in a POM file according to MAVEN coordinates to load the JAR package.
In the step 2, initializing an fdruiddataSource class, configuring and generating a data source connection object, and calling a getConnection () method of the fdruiddataSource object to obtain an available middleware instance connection in the database middleware cluster.
The step 4 specifically includes:
step 4-1: the client randomly selects a data connection object from an available data source set at regular time, extracts a connection parameter to request to access a database middleware cluster, acquires a middleware instance in an online state in the current database middleware cluster, and records the name of the middleware instance;
step 4-2: comparing the recorded name of the middleware instance with a data source connection object of a client available data source set, and screening out the middleware instance of a specified type, wherein the middleware instance of the specified type comprises an online instance which is not recorded with the available data source set and a offline instance which is recorded with the available data source set;
step 4-3: aiming at the online instance which is not recorded with the available data source set, acquiring information of the middleware instance, generating connection parameters of the middleware instance, creating a data connection object based on the connection parameters and storing the data connection object into the available data source set; for the offline instance with the recorded available data source set, moving the data connection object corresponding to the middleware instance out of the available data source set, and transferring the data connection object into the data source set to be offline;
step 4-4: tracking a data source connection object newly entering a data source set to be offline, and checking whether an ongoing data request task still exists in the connection between the database middleware client and the middleware instance; if the data request task exists, after the data request task is executed, closing the connection between the database middleware client and the middleware example; if not, the connection between the database middleware client and the middleware instance is directly closed.
In the step 5, before the client requests to connect the middleware instance, comparing the sizes of the client connection number and the maximum connection number; when the number of client connections is equal to the maximum number of connections, the connections with the middleware instance need to wait until other connections are finished; and when the client connection number is less than the maximum connection number, sending a connection request to the corresponding middleware instance.
In the step 5, if the service system frequently fails to connect the middleware instance through the client and the continuous times reach the preset threshold, the client moves the data source connection object corresponding to the middleware instance out of the available data source set and moves the data source connection object into the data source set to be offline; the database middleware client redistributes a new data source connection object to the service system according to the configured load balancing algorithm again to establish a new connection with the middleware instance and request to access the database.
In the step 5, the load balancing algorithm used by the client includes a polling method, a random method, a weighted polling method, a weighted random method, a source address hashing method, and a minimum connection method.
The load balancing algorithm in the step 5 preferably selects a minimum connection algorithm, obtains online middleware instances, calculates the number of connections between each middleware instance and the client, and obtains the first 5 middleware instances in the descending order of the number of connections; calculating the average response time of the 5 middleware examples, and selecting the middleware example with the shortest average response time as the preferred middleware example; the average response time is the time average of the middleware instance processing the service request.
The invention achieves the following beneficial effects:
(1) the method and the system have the advantages that the client JAR packet of the database middleware cluster is introduced into the service application, the available data source set for monitoring the available state of the middleware instance and the data source set to be offline are configured, data communication based on the JDBC protocol is realized, the problems of connection, load, abnormity, fusing and the like between the client JAR packet and the database middleware cluster can be automatically processed, and the trouble of configuring load balancing hardware and software at a server is reduced;
(2) the invention periodically compares each middleware instance in the database middleware cluster with the middleware instances in the available data source set, eliminates the middleware instances which are offline or have abnormity in time, rejoins the new instances or the instances which are recovered to be online, improves the efficiency of data source connection, and can effectively solve the problem of processing unfinished service requests when the middleware instances are offline.
Drawings
FIG. 1 is a flowchart of a method for balancing client loads based on a database middleware cluster according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a client load balancing system based on a database middleware cluster according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a connection between a service system and an example middleware according to an embodiment of the present invention;
FIG. 4 is a timing diagram illustrating the database middleware client initiation and request processing according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of middleware instance maintenance based on a database middleware client in the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clear, the following detailed description describes the client initialization process and the maintenance and selection of the database middleware cluster instance.
Fig. 1 is a flowchart of a load balancing method for a high-availability client based on a database middleware cluster in an embodiment of the present invention, which specifically includes:
step 1: acquiring a first connection credential of a client and a database middleware cluster: compiling a client JAR package of the database middleware in Java language, and importing the client JAR package into an application; in the process that a client JAR packet executes registration driving in an application, a middleware instance is randomly selected from a database middleware cluster, and the information of the middleware instance is extracted and configured into initial connection parameters to be used as a first connection evidence of the client and the database middleware cluster;
the connection parameters comprise a middleware instance URL address, an account, a password, an IP and port number, a maximum connection number and a minimum connection number; the maximum connection number is used for limiting the maximum number of middleware instances to which the client establishes connection; the minimum connection number is used for limiting the minimum number of middleware instances with which the client maintains connection; the state of the middleware instance comprises an online state and a offline state; the middleware instance is on line, the availability of the middleware instance is represented, and the client can be connected with the database middleware cluster through the instance; the offline of the middleware instance represents that the middleware instance is unavailable, and the client cannot be connected with the database middleware cluster through the instance;
in the embodiment of the invention, the JAR package of the client is put into a specified lib directory for loading, or the following codes are written in a POM file for loading according to the MAVEN coordinate;
step 2: generating a data source connection object by using the first connection certificate and storing the data source connection object into the client: creating an available data source set and a data source set to be offline in the client, extracting the IP and the port number of the middleware example from the initial connection parameters in the step 1, generating a default data source connection object, naming the data source connection object by the name of the middleware example, and storing the data source connection object into the available data source set; the data source connection object and the middleware instance keep a one-to-one correspondence relationship; in the embodiment of the invention, a data source connection object is configured and generated by initializing an FDruidDataSource class, and a getConnection () method of the FDruidDataSource object is called to obtain an available middleware instance connection taken to a database middleware cluster. Code reference is as follows
FDruidDataSource druidDataSource＝new FDruidDataSource()；
druidDataSource.setClientName("")；
druidDataSource.setDriverClassName("com.mysql.cj.jdbc.Driver")；
druidDataSource.setUrl(“”)；
druidDataSource.setUsername(“”)；
druidDataSource.setPassword(“”)；
druidDataSource.setInitialSize(5)；
druidDataSource.setMaxWait(5000)；
druidDataSource.setMaxActive(100)；
druidDataSource.setValidationQuery("select 1from dual")；
And step 3: acquiring all online middleware instances from a database middleware cluster: the client acquires a default data source connection object from the available data source set, determines a pre-connected middleware instance according to the IP and the port number in the data source connection object, extracts a URL (Uniform resource locator) address in initial connection parameters, accesses a database middleware cluster by a JDBC (joint discovery code) protocol and is connected with the middleware instance corresponding to the data source connection object; monitoring that the connection between the client and the middleware instance is successful, and adding 1 to the connection number of the client; simultaneously requesting to acquire all online middleware instances and instance information in a current database middleware cluster, generating connection parameters of the middleware instances, creating a data source connection object corresponding to the instances at a client, and putting the data source connection object into an available data source set of the client;
and 4, step 4: after the first connection in the step 3 is successful, periodically checking the middleware instance state in the database middleware cluster, and reallocating the data source connection objects in the available data source set and the data source connection objects in the data source set to be offline of the client according to the middleware instance state; the method comprises the following specific steps:
step 4-1: the client randomly selects a data connection object from an available data source set at regular time, extracts a connection parameter to request to access a database middleware cluster, acquires a middleware instance in an online state in the current database middleware cluster, and records the name of the middleware instance;
step 4-2: comparing the recorded name of the middleware instance with a data source connection object of a client available data source set, and screening out 2 types of middleware instances, wherein the 2 types of middleware instances comprise (1) online instances which are not recorded with the available data source set; (2) an offline instance of the logged set of available data sources;
step 4-3: acquiring information of the middleware instance aiming at the online instance which is not recorded with the available data source set, generating a connection parameter of the middleware instance according to the step 1, and creating a data connection object based on the connection parameter and storing the data connection object into the available data source set; for the offline instance with the recorded available data source set, moving the data connection object corresponding to the middleware instance out of the available data source set, and transferring the data connection object into the data source set to be offline;
step 4-4: tracking a data source connection object newly entering a data source set to be offline, and checking whether an ongoing data request task still exists in the connection between the database middleware client and the middleware instance; if the data request task exists, after the data request task is executed, closing the connection between the database middleware client and the middleware example; if not, directly closing the connection between the database middleware client and the middleware instance; and after the connection between the client and the middleware instance is monitored to be closed, subtracting 1 from the connection number of the client, and clearing the data source connection object from the data source set to be offline.
And 5: the application system acquires a data source connection object by using a load balancing algorithm, and a connection middleware instance sends a service request, wherein the specific flow comprises the following steps:
step 5-1: the application system generates a database access request, and a database middleware client installed in the application system acquires the IP and the port number in a data source connection object according to a polling method and the sequence of the middleware instances entering an available data source set;
step 5-2: analyzing the URL address and the maximum connection number of the connection middleware example from the connection parameters; checking whether the current client connection number reaches the maximum connection number, if so, entering the current connection request into a waiting queue until the client connection number is smaller than the maximum connection number, and requesting to connect the middleware instance; if the number of the connections is less than the maximum number of the connections, continuing to execute the step 5-3;
step 5-3: accessing the middleware cluster of the database according to the URL address, and requesting connection with the middleware instance according to the IP and the port number; if the connection is successful, the middleware instance acquires data from the database cluster according to the service request and returns the data to the client; if the connection fails, continuing to request the connection, and synchronously counting the connection failure times;
step 5-4: if the continuous requests fail and the failure times reach a preset threshold value, the database middleware client moves the data source connection object corresponding to the middleware instance out of the available data source set and moves the data source connection object into a to-be-offline data source set; and the client continuously selects the next data source connection object to the service system according to the sequence of the middleware example entering the available data source set, and continuously initiates a connection request for connecting the middleware cluster of the database according to the step 5-2-the step 5-4 until the request is processed.
Fig. 2 is a schematic structural diagram of a high-availability client load balancing system based on a database middleware cluster in an embodiment of the present invention, including a service system, a database middleware cluster, and a database cluster that are connected in sequence.
Configuring a client in a service system for controlling the connection with a middleware instance in a database middleware cluster; the business system is connected with a database middleware cluster through a client, and the database cluster consists of not less than 2 databases; the database middleware cluster consists of not less than 2 middleware instances, and the state of each middleware instance comprises an online state and an offline state;
an available data source set and a data source set to be offline are arranged in the client, wherein the available data source set is used for storing the middleware instances in an online state, which are obtained from the database middleware cluster; the data source set to be offline is used for storing the middleware instances in the offline state in the database middleware cluster; when a user generates a data request in the service system, any data source connection object in the available data source set is called to connect the corresponding middleware instance in the database middleware cluster.
As shown in fig. 3, a user obtains an instance a data source through a client, and connects to a corresponding middleware instance a in a database middleware cluster; and connecting the corresponding middleware instance C in the database middleware cluster through the client instance C data source.
Fig. 4 is a flowchart of a timing procedure for starting and requesting a database cluster by a middleware client in a database cluster according to an embodiment of the present invention, which includes the following specific steps:
step 401: acquiring a cluster instance, and returning an instance set: in the embodiment of the invention, a middleware instance A and a middleware instance B are taken as columns, when a client is initialized, a default instance A data source is generated according to configuration, an instance A in a database middleware cluster is accessed through a JDBC protocol, if the instance A and the instance B in the database middleware cluster are online at the same time, the instance A and the instance B are acquired at the same time, and an instance B data source is created and put into a normal data source queue, as shown in FIG. 5;
step 402: starting a timing task, and maintaining a normal data source queue: referring to fig. 5, the client selects a data source connection of an instance to obtain the latest online instance in the database middleware cluster. At this time, the instance A and the instance C are online, so that the client can maintain the data source of the instance A unchanged, move the data source of the instance B into the data source queue to be offline, create the data source of the instance C and place the data source into the normal data source queue, if the instance selected by the client has problems and cannot normally acquire information, the client can move the data source of the instance into the data source queue to be offline and reselect a new instance after the failure times reach the preset value, and then continue to execute the step 402;
step 403: starting a timing task, maintaining a data source queue to be offline, and after checking that the data source of the instance B does not request a task, closing and removing the data source resource of the instance B by the client, as shown in FIG. 5;
step 404: the service system acquires the data source connection of a middleware instance according to the algorithm; in the embodiment of the invention, taking the middleware example C as an example, a user acquires a data source connection of an available middleware example C through a client by means of a service system, the data source connection is communicated with a database middleware cluster, if the service system continuously fails to use the connection communication for multiple times and reaches a fusing condition, the client moves the data source of the example into a data source queue to be offline, and a new data source of the example is redistributed to be connected to the service system.
Step 405: sending a service request, and acquiring service data from a database cluster: as shown in fig. 5, a client generates a service request in a service system, sends the service request to a database middleware cluster through the client, and accesses the database cluster to obtain data through a middleware instance C in the database middleware cluster; and returning the data returned by the database cluster to the client through the middleware instance C.
The state information of the middleware cluster is obtained through the JDBC protocol, the state information is different from a database cluster in the prior art, the database is directly connected after the database information is obtained through the middleware, the load and the route are performed on the middleware cluster, and the parameter configuration scheme with excellent effect is finally obtained according to the conditions of actual machine configuration, service flow and the like and the dynamic adjustment of the detected data after the operation for a period of time because the maximum connection number and the minimum connection number configured by the method in the prior art are not optimally set.
The invention achieves the following beneficial effects:
(1) the method and the system have the advantages that the client JAR packet of the database middleware cluster is introduced into the service application, the available data source set for monitoring the available state of the middleware instance and the data source set to be offline are configured, data communication based on the JDBC protocol is realized, the problems of connection, load, abnormity, fusing and the like between the client JAR packet and the database middleware cluster can be automatically processed, and the trouble of configuring load balancing hardware and software at a server is reduced;
(2) the invention periodically compares each middleware instance in the database middleware cluster with the middleware instances in the available data source set, eliminates the middleware instances which are offline or have abnormity in time, rejoins the new instances or the instances which are recovered to be online, improves the efficiency of data source connection, and can effectively solve the problem of processing unfinished service requests when the middleware instances are offline.
The above embodiments do not limit the present invention in any way, and all other modifications and applications that can be made to the above embodiments in equivalent ways are within the scope of the present invention.
Claims (9)
1. A high-availability client load balancing method based on a database middleware cluster is characterized by comprising the following steps:
step 1: the loading of the client side obtains the first connection credential of the client side and the database middleware cluster, and specifically comprises the following steps: compiling a JAR package of a client in Java language, wherein the client is a client of the database middleware, and importing the JAR package of the client into an application; in the process that a JAR packet of a client executes registration driving in an application, a middleware instance is arbitrarily selected from a database middleware cluster, and information of the middleware instance is configured as an initial connection parameter, wherein the information is used as a first connection evidence of the client and the database middleware cluster; the initial connection parameters include: middleware instance URL, account, password, IP and port number, maximum connection number, minimum connection number; the maximum connection number is used for limiting a client to establish connection with a maximum number of middleware instances; the minimum connection number is used for limiting how many intermediate instances a client at least keeps establishing connection with; if the number of the current client side connection middleware instances exceeds the preset maximum connection number, the connection request initiated by the client side needs to wait; the state of the middleware instance comprises an online state and a offline state; the middleware instance is on line, the availability of the middleware instance is represented, and the client can be connected with the database middleware cluster through the instance; the offline of the middleware instance represents that the middleware instance is unavailable, and the client cannot be connected with the database middleware cluster through the instance;
step 2: the method comprises the following steps of generating a data source connection object by using a first connection certificate and storing the data source connection object into a client, specifically: creating an available data source set and a data source set to be offline in the client, extracting the IP and the port number of the middleware instance according to the initial connection parameters obtained in the step 1, generating a default data source connection object and storing the default data source connection object in the available data source set; the data source connection object and the middleware instance keep a one-to-one correspondence relationship; the available data source set is used for storing the middleware instances in an online state, which are obtained from the database middleware cluster; the data source set to be offline is used for storing the middleware instances in the offline state in the database middleware cluster;
and step 3: acquiring all online middleware instances from a database middleware cluster: the client acquires a default data source connection object from the available data source set, determines a pre-connected middleware instance according to the IP and the port number in the data source connection object, extracts a URL (Uniform resource locator) address in initial connection parameters, accesses a database middleware cluster by a JDBC (joint discovery code) protocol and is connected with the middleware instance corresponding to the data source connection object; simultaneously requesting to acquire all online middleware instances and instance information in a current database middleware cluster, generating connection parameters of the middleware instances, creating a data source connection object corresponding to the instances at a client, and putting the data source connection object into an available data source set of the client;
and 4, step 4: after the first connection in the step 3 is successful, periodically checking the middleware instance state in the database middleware cluster, and reallocating the data source connection objects in the available data source set and the data source connection objects in the data source set to be offline of the client according to the middleware instance state;
and 5: the service system obtains a connection middleware example according to a configured load balancing algorithm: the service system generates a database access request, triggers a database middleware client in the service system to select a data source connection object according to a preset load balancing algorithm, analyzes connection parameters and requests to access a middleware instance of a corresponding data source connection object in a database middleware cluster according to the connection parameters; before the client establishes connection with the middleware instance, the connection of the client is confirmed to not reach the maximum connection number, and if the connection is reached, the connection request needs to wait for post-processing.
2. The method for load balancing of the high-availability clients based on the middleware cluster of the database as claimed in claim 1, wherein: in the step 1, in the connection process between the client and the middleware instance, the connection condition between the middleware client and the middleware instance is monitored in real time, and the number of the middleware instances which are successfully connected with the client at present is counted according to the number of the client connections; when the client is successfully connected with the middleware instance through the data source connection object, adding 1 to the client connection number; when the client's connection to the middleware instance ends, the client connection number is decremented by "1".
3. The method for load balancing of the high-availability clients based on the middleware cluster of the database as claimed in claim 2, wherein: in step 1, the loading method of the client includes: for the Java project, loading the JAR package of the client side by putting the JAR package into a specified lib directory, and for the MAVEN project, writing a loading code in a POM file according to MAVEN coordinates to load the JAR package.
4. The method for load balancing of the high-availability clients based on the middleware cluster of the database as claimed in claim 3, wherein: in the step 2, initializing an fdruiddataSource class, configuring and generating a data source connection object, and calling a getConnection () method of the fdruiddataSource object to obtain an available middleware instance connection in the database middleware cluster.
5. The method for load balancing of the high-availability clients based on the middleware cluster of the database as claimed in claim 4, wherein: the step 4 specifically includes:
step 4-1: the client randomly selects a data connection object from an available data source set at regular time, extracts a connection parameter to request to access a database middleware cluster, acquires a middleware instance in an online state in the current database middleware cluster, and records the name of the middleware instance;
step 4-2: comparing the recorded name of the middleware instance with a data source connection object of a client available data source set, and screening out the middleware instance of a specified type, wherein the middleware instance of the specified type comprises an online instance which is not recorded with the available data source set and a offline instance which is recorded with the available data source set;
step 4-3: aiming at the online instance which is not recorded with the available data source set, acquiring information of the middleware instance, generating connection parameters of the middleware instance, creating a data connection object based on the connection parameters and storing the data connection object into the available data source set; for the offline instance with the recorded available data source set, moving the data connection object corresponding to the middleware instance out of the available data source set, and transferring the data connection object into the data source set to be offline;
step 4-4: tracking a data source connection object newly entering a data source set to be offline, and checking whether an ongoing data request task still exists in the connection between the database middleware client and the middleware instance; if the data request task exists, after the data request task is executed, closing the connection between the database middleware client and the middleware example; if not, the connection between the database middleware client and the middleware instance is directly closed.
6. The method for load balancing of the high-availability clients based on the middleware cluster of the database as claimed in claim 5, wherein: in the step 5, before the client requests to connect the middleware instance, comparing the sizes of the client connection number and the maximum connection number; when the number of client connections is equal to the maximum number of connections, the connections with the middleware instance need to wait until other connections are finished; and when the client connection number is less than the maximum connection number, sending a connection request to the corresponding middleware instance.
7. The method for load balancing of high-availability clients based on the cluster of the database middleware as claimed in claim 6, wherein: in the step 5, if the service system frequently fails to connect the middleware instance through the client and the continuous times reach the preset threshold, the client moves the data source connection object corresponding to the middleware instance out of the available data source set and moves the data source connection object into the data source set to be offline; the database middleware client redistributes a new data source connection object to the service system according to the configured load balancing algorithm again to establish a new connection with the middleware instance and request to access the database.
8. The method for load balancing of high-availability clients based on the cluster of the database middleware as claimed in claim 7, wherein: in the step 5, the load balancing algorithm used by the client includes a polling method, a random method, a weighted polling method, a weighted random method, a source address hashing method, and a minimum connection method.
9. The method for load balancing of high-availability clients based on the cluster of the database middleware as claimed in claim 8, wherein: the load balancing algorithm in the step 5 preferably selects a minimum connection algorithm, obtains online middleware instances, calculates the number of connections between each middleware instance and the client, and obtains the first 5 middleware instances in the descending order of the number of connections; calculating the average response time of the 5 middleware examples, and selecting the middleware example with the shortest average response time as the preferred middleware example; the average response time is the time average of the middleware instance processing the service request.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110439382.8A CN113032431B (en) | 2021-04-23 | 2021-04-23 | High-availability client load balancing method based on database middleware cluster |
| PCT/CN2022/075644 WO2022222579A1 (en) | 2021-04-23 | 2022-02-09 | Database middleware cluster-based high-availability client load balancing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110439382.8A CN113032431B (en) | 2021-04-23 | 2021-04-23 | High-availability client load balancing method based on database middleware cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113032431A true CN113032431A (en) | 2021-06-25 |
| CN113032431B CN113032431B (en) | 2022-02-18 |
Family
ID=76457820
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110439382.8A Active CN113032431B (en) | 2021-04-23 | 2021-04-23 | High-availability client load balancing method based on database middleware cluster |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113032431B (en) |
| WO (1) | WO2022222579A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022222579A1 (en) * | 2021-04-23 | 2022-10-27 | 焦点科技股份有限公司 | Database middleware cluster-based high-availability client load balancing method |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115550424B (en) * | 2022-12-02 | 2023-03-14 | 苏州万店掌网络科技有限公司 | Data caching method, device, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105224637A (en) * | 2015-09-24 | 2016-01-06 | 珠海许继芝电网自动化有限公司 | A kind of based on PostgreSQL database active and standby/the comprehensive method of cluster application |
| CN106453564A (en) * | 2016-10-18 | 2017-02-22 | 北京京东尚科信息技术有限公司 | Elastic cloud distributed massive request processing method, device and system |
| CN106453125A (en) * | 2016-11-04 | 2017-02-22 | 中国电子科技集团公司第二十八研究所 | Load balancing system for remote service calling based on real-time load rate |
| CN106570074A (en) * | 2016-10-14 | 2017-04-19 | 深圳前海微众银行股份有限公司 | Distributed database system and implementation method thereof |
| US20180293275A1 (en) * | 2017-04-10 | 2018-10-11 | Sap Se | Massively parallel processing database middleware connector |
| CN110297862A (en) * | 2019-07-04 | 2019-10-01 | 中国联合网络通信集团有限公司 | Data bank access method and Database-access Middleware Based |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101169785A (en) * | 2007-11-21 | 2008-04-30 | 浪潮电子信息产业股份有限公司 | Clustered database system dynamic loading balancing method |
| US10594779B2 (en) * | 2015-08-27 | 2020-03-17 | Datometry, Inc. | Method and system for workload management for data management systems |
| CN108595574B (en) * | 2018-04-16 | 2021-11-02 | 上海达梦数据库有限公司 | Database cluster connection method, device, equipment and storage medium |
| US11038952B2 (en) * | 2019-07-12 | 2021-06-15 | Ebay Inc. | Connection service discovery and load rebalancing |
| CN110727709A (en) * | 2019-10-10 | 2020-01-24 | 北京优炫软件股份有限公司 | Cluster database system |
| CN111277629A (en) * | 2020-01-13 | 2020-06-12 | 浙江工业大学 | High-availability-based web high-concurrency system and method |
| CN111538605B (en) * | 2020-04-24 | 2023-04-11 | 北京思特奇信息技术股份有限公司 | Distributed data access layer middleware and command execution method and device |
| CN113032431B (en) * | 2021-04-23 | 2022-02-18 | 焦点科技股份有限公司 | High-availability client load balancing method based on database middleware cluster |
-
2021
- 2021-04-23 CN CN202110439382.8A patent/CN113032431B/en active Active
-
2022
- 2022-02-09 WO PCT/CN2022/075644 patent/WO2022222579A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105224637A (en) * | 2015-09-24 | 2016-01-06 | 珠海许继芝电网自动化有限公司 | A kind of based on PostgreSQL database active and standby/the comprehensive method of cluster application |
| CN106570074A (en) * | 2016-10-14 | 2017-04-19 | 深圳前海微众银行股份有限公司 | Distributed database system and implementation method thereof |
| CN106453564A (en) * | 2016-10-18 | 2017-02-22 | 北京京东尚科信息技术有限公司 | Elastic cloud distributed massive request processing method, device and system |
| CN106453125A (en) * | 2016-11-04 | 2017-02-22 | 中国电子科技集团公司第二十八研究所 | Load balancing system for remote service calling based on real-time load rate |
| US20180293275A1 (en) * | 2017-04-10 | 2018-10-11 | Sap Se | Massively parallel processing database middleware connector |
| CN110297862A (en) * | 2019-07-04 | 2019-10-01 | 中国联合网络通信集团有限公司 | Data bank access method and Database-access Middleware Based |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022222579A1 (en) * | 2021-04-23 | 2022-10-27 | 焦点科技股份有限公司 | Database middleware cluster-based high-availability client load balancing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113032431B (en) | 2022-02-18 |
| WO2022222579A1 (en) | 2022-10-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10122595B2 (en) | System and method for supporting service level quorum in a data grid cluster | |
| US9083614B2 (en) | System and method for supporting out-of-order message processing in a distributed data grid | |
| US9930111B2 (en) | Techniques for web server management | |
| US7185096B2 (en) | System and method for cluster-sensitive sticky load balancing | |
| JP4637842B2 (en) | Fast application notification in clustered computing systems | |
| US8191068B2 (en) | Resource management system, resource information providing method and program | |
| CN113032431B (en) | High-availability client load balancing method based on database middleware cluster | |
| US20080091806A1 (en) | Dynamic On-Demand Clustering | |
| US8447757B1 (en) | Latency reduction techniques for partitioned processing | |
| CN110912972B (en) | Service processing method, system, electronic equipment and readable storage medium | |
| CN110932912A (en) | Method for realizing unified management of configuration files under micro-service architecture | |
| JP2007506157A (en) | Hierarchical management of dynamic resource allocation in multi-node systems | |
| CN110727508A (en) | Task scheduling system and scheduling method | |
| US11138215B2 (en) | Method and system for implementing parallel database queries | |
| EP3672203A1 (en) | Distribution method for distributed data computing, device, server and storage medium | |
| JP2016177324A (en) | Information processing apparatus, information processing system, information processing method, and program | |
| CN112631756A (en) | Distributed regulation and control method and device applied to space flight measurement and control software | |
| US10715608B2 (en) | Automatic server cluster discovery | |
| CN116055499A (en) | Method, equipment and medium for intelligently scheduling cluster tasks based on redis | |
| CN114168312A (en) | Distributed cluster load balancing method and device and storage medium | |
| CN113630317A (en) | Data transmission method and device, nonvolatile storage medium and electronic device | |
| US20230362103A1 (en) | Reducing placement conflicts between concurrent virtual machine allocations | |
| US11586632B2 (en) | Dynamic transaction coalescing | |
| CN108173775A (en) | For the method and system of server current limliting | |
| JP2023537703A (en) | Randomize heartbeat communication across multiple partition groups |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |