CN111818147B - System and method for realizing automatic load balancing of openstack swift service - Google Patents
System and method for realizing automatic load balancing of openstack swift service Download PDFInfo
- Publication number
- CN111818147B CN111818147B CN202010619780.3A CN202010619780A CN111818147B CN 111818147 B CN111818147 B CN 111818147B CN 202010619780 A CN202010619780 A CN 202010619780A CN 111818147 B CN111818147 B CN 111818147B
- Authority
- CN
- China
- Prior art keywords
- service
- node
- load balancing
- storage service
- authentication
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/101—Server selection for load balancing based on network conditions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a system for realizing automatic load balancing of openstack swift service, which comprises a client, a service registration and authentication node, a load balancing plug-in module, an acquisition monitoring module, a storage service gateway proxy node and a storage node, wherein the client is connected with the client through a network; the load balancing plug-in module and the acquisition monitoring module form a control link, and the load balancing plug-in module is used for modifying and setting a terminal address of a distributed object storage service in the service registration and authentication node or a terminal address of a server object storage service in the service registration and authentication node according to a balancing rule and monitoring state information of the acquisition monitoring module; the acquisition monitoring module is used for monitoring the state of the proxy node server of the storage service gateway and feeding the monitored state information back to the load balancing plug-in module of the service registration and authentication node. The invention has the advantages that: the openstack swift service automatic load balancing is realized, the equipment utilization efficiency is improved, and the maintenance cost is greatly saved.
Description
Technical Field
The invention relates to the technical field of network communication, in particular to a system and a method for realizing automatic load balancing of openstack swift service.
Background
The Openstack swift provides distributed object storage service, and a service cluster consists of three types of nodes: service registration and authentication node, proxy node, storage node. The service registration and authentication node provides an access entrance of the object storage service and user authentication login, the proxy node is used as a service gateway responsible for the object storage service, and the storage node is responsible for data storage. The client obtains the endpoint of the swift service through the keystone, then establishes connection with the proxy, and uploads data to the storage node through the proxy or downloads data from the storage node to the client. And 2 proxy nodes are configured into a high-availability cluster to realize high availability of the service. This deployment architecture is only suitable for smaller scale application scenarios. For high-throughput scenarios with high-speed, proxy is obviously a bottleneck. Therefore, in a part of production environments, 1 hardware load balancer is deployed in front of a plurality of proxy nodes to meet the needs of a high-concurrency service scene, but the deployment scheme of the hardware load balancer is added in the prior art, but the hardware load balancer can only meet the needs of the high-concurrency service scene to a certain extent, the load balancer itself can become the bottleneck of the performance of the whole storage cluster, and the service capability of the whole storage cluster is not released fundamentally. Meanwhile, due to the addition of the devices, the construction cost and the maintenance cost of the whole storage cluster are increased. The storage service gateway proxy in the prior art cannot realize automatic load balancing, and neither a proxy HA scheme nor a hardware load balancer is added to solve the performance bottleneck problem of the storage gateway fundamentally and in a high cost performance manner.
Disclosure of Invention
Aiming at the problems of load balancing bottleneck and high maintenance cost in the prior art, the invention provides a system and a method for realizing automatic load balancing of openstack swift service.
The invention discloses a system for realizing automatic load balancing of openstack swift service, which comprises a client, a service registration and authentication node, a load balancing plug-in module, an acquisition monitoring module, a storage service gateway proxy node and a storage node, wherein the client is connected with the client through a network;
the client is a terminal used by a user for uploading and downloading data, and the client is used for establishing connection with a proxy node of the storage service gateway and uploading or downloading data;
the service registration and authentication node is a node for a user to perform authentication login, the user accesses the storage service through the service registration and authentication node and performs authentication login, and the service registration and authentication node is an access entry for accessing the storage service;
the load balancing plug-in module is arranged in the service registration and authentication node, the load balancing plug-in module and the acquisition monitoring module form a control link, and the load balancing plug-in module is used for modifying and setting a terminal address of a distributed object storage service in the service registration and authentication node or a terminal address of a server object storage service in the service registration and authentication node according to a balancing rule and monitoring state information of the acquisition monitoring module;
a link is established between the acquisition monitoring module and the storage service gateway proxy node, and the acquisition monitoring module is used for monitoring the state of a storage service gateway proxy node server and feeding the monitored state information back to a load balancing plug-in module of the service registration and authentication node;
the storage service gateway proxy node is a service gateway responsible for object storage service, and is used for establishing connection with a client and storing or downloading data between the storage service gateway proxy node and the storage node;
the storage nodes are used for storing data.
Further, the balancing rule is to modify the terminal address in the service registration and authentication node in a polling manner according to the storage service gateway proxy node server IP address list.
Further, the load balancing plug-in module modifies the terminal address of the distributed object storage service in the service registration and authentication node in a polling mode by using the storage service gateway proxy node server IP address list to realize load balancing.
Further, the balancing rule is to set the terminal address of the server object storage service in the service registration and authentication node according to the network bandwidth occupation condition of the storage service gateway proxy node server.
Further, the load balancing plug-in module sets the terminal address of the server object storage service in the service registration and authentication node according to the network bandwidth occupation condition of the storage service gateway proxy node server.
Furthermore, the load balancing plug-in module sets the weight and the sequence of each storage service gateway proxy node server in the proxy server IP address list according to the bandwidth network state of each storage service gateway proxy node server to realize load balancing.
Further, the collection monitoring module is arranged at the service registration and authentication node.
The other purpose of the invention is to provide a method for realizing automatic load balancing of openstack swift service, an authentication link is established between a client and service registration and authentication nodes, the client acquires a terminal address of distributed object storage service through the authentication link, the client establishes a transmission link with a proxy server of a designated storage service gateway proxy node according to the acquired terminal address, a load balancing link is established between the authentication nodes and each storage service gateway proxy node, the servers of each storage service gateway proxy node are balanced, processed, collected and monitored through the load balancing link, the service registration and authentication nodes modify and set the terminal address of the distributed object storage service inside the service registration and authentication nodes or the terminal address of the server object storage service in the service registration and authentication nodes according to the results of the balancing processing and the collected and monitored, the equalization process and acquisition monitoring process are implemented by a system as claimed in claim 1.
Further, the balancing process is configured to modify the terminal address in the service registration and authentication node in a polling manner according to the IP address list of the proxy node server of the storage service gateway.
Further, the balancing process is set to set the terminal address of the server object storage service in the service registration and authentication node according to the network bandwidth occupation condition of the storage service gateway proxy node server.
Compared with the prior art, the invention has the beneficial effects that:
according to the invention, by the scheme of setting the monitoring module and setting the load balancing plug-in the service registration and authentication node, the automatic load balancing of the openstack swift proxy is realized, the utilization efficiency of the equipment is improved, all the storage service gateway proxy node servers are in an effective working state, the overall construction cost and the maintenance cost are effectively reduced, no additional hardware equipment is required, and an HA solution is not introduced.
Drawings
FIG. 1 is a diagram of a system link relationship architecture in accordance with the present invention;
fig. 2 is a functional block diagram of the system of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
The invention is described in further detail below with reference to the attached drawing figures:
referring to fig. 1 and fig. 2, the present embodiment discloses a system for implementing automatic load balancing of openstack swift service, including a client, a service registration and authentication node, a load balancing plug-in module, an acquisition monitoring module, a storage service gateway proxy node, and a storage node.
The Keystone can be used as an authentication system of a service registration and authentication node, the endpoint is a terminal address, and the proxy gateway is a proxy node gateway. In the specific embodiment, the automatic load balancing of the multiple storage gateways proxy is realized under the condition that no additional hardware equipment is added, and the high availability of the service is ensured. The system adopts Keystone as an authentication system, a plurality of endpoints are added under the swift service, and one proxy gateway is one endpoint. Keystone certification, although supporting multiple-endpoint, multiple-endpoint cannot achieve load balancing. The end point-based load balancing function is achieved by adding a load balancing plug-in of Keystone, namely plugin in FIG. 1.
The process of load balancing is transparent to the client, that is, the client, namely the APP used by the user, acquires the Endpoint of the distributed object storage service through the link 1 in fig. 1, and then connects the designated swift proxy server according to the acquired Endpoint to upload and download data. The load balancing logic process is mainly completed by the cooperative work of the acquisition monitoring and the Keystone load balancing plug-in, two load balancing modes of polling and network states are supported, and the whole system can only work in one load balancing mode.
Polling load balancing, and modifying the Endpoint of the distributed object storage service in the Keystone by using a swift proxy server, namely an IP address list of a storage service gateway proxy node in a polling mode by using a Keystone load balancing plug-in, thereby realizing load balancing. And the acquisition monitoring module continuously monitors the states of all the swift proxy servers through a link a and feeds the state information back to the Keystone plug-in unit through a link b. When a certain swift proxy server is abnormal in service, the Keystone plug-in can remove the abnormal service from the proxy server IP address list so as to ensure the availability of the service.
And based on the load balancing of the network state, setting the endPoint of the server object storage service in the Keystone according to the network bandwidth occupation condition of each swift proxy server so as to achieve the purpose of load balancing. And the acquisition monitoring module continuously monitors the states of all the swift proxy servers through a link a and feeds the state information back to the Keystone load balancing plug-in unit through a link b. And the Keystone load balancing plug-in sets the weight and the sequence of each proxy server in the proxy server IP address list according to the bandwidth network state of each proxy server generation so as to achieve the purpose of load balancing.
Specifically, the client in this embodiment is an APP used by a user, and is used for a terminal for uploading and downloading data, and the client is used for establishing a connection with a storage service gateway proxy node and uploading or downloading data; the service registration and authentication node is a node for a user to perform authentication login, the user accesses the storage service and performs authentication login through the service registration and authentication node, the service registration and authentication node is an access entry for accessing the storage service, and in the specific embodiment, the service registration and authentication node can be an authentication system; the load balancing plug-in module is arranged in the service registration and authentication node, the load balancing plug-in module and the acquisition monitoring module form a control link, and the load balancing plug-in module is used for modifying and setting a terminal address of a distributed object storage service in the service registration and authentication node or a terminal address of a server object storage service in the service registration and authentication node according to a balancing rule and monitoring state information of the acquisition monitoring module; a link is established between the acquisition monitoring module and the storage service gateway proxy node, and the acquisition monitoring module is used for monitoring the state of a storage service gateway proxy node server and feeding the monitored state information back to a load balancing plug-in module of the service registration and authentication node; the storage service gateway proxy node is a service gateway responsible for object storage service, and is used for establishing connection with a client and storing or downloading data between the storage service gateway proxy node and the storage node; the storage nodes are used for storing data, and are storage devices distributed at various nodes.
Preferably, the collection monitoring module is disposed at the service registration and authentication node, and may be disposed at other locations in other embodiments of the present invention.
According to the invention, the acquisition monitoring module is arranged to sense the swift proxy running state in real time, so that load balance and effectiveness are ensured. By setting a keystone load balancing plug-in, the module enables the client to obtain an ideal service URL in an endpoint updating mode, so that a load function is realized. Through the keystone and the operation framework of the acquisition monitoring module, according to the actual operation load of a keystone node, the acquisition monitoring module can be deployed together with the keystone and can also be separately deployed, and the setting is convenient.
By the scheme of setting the monitoring module and setting the load balancing plug-in the service registration and authentication node, automatic load balancing of openstack swift proxy is realized, the utilization efficiency of equipment is improved, all the storage service gateway proxy node servers are in effective working states, the overall construction cost and the maintenance cost are effectively reduced, and an HA solution is not required to be added or introduced.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (5)
1. A system for realizing automatic load balancing of openstack swift service is characterized by comprising a client, a service registration and authentication node, a load balancing plug-in module, an acquisition monitoring module, a storage service gateway proxy node and a storage node;
the client is a terminal used by a user for uploading and downloading data, and the client is used for establishing connection with a proxy node of the storage service gateway and uploading or downloading data;
the service registration and authentication node is a node for a user to perform authentication login, the user accesses the storage service through the service registration and authentication node and performs authentication login, and the service registration and authentication node is an access entry for accessing the storage service;
the load balancing plug-in module is arranged in the service registration and authentication node, the load balancing plug-in module and the acquisition monitoring module form a control link, and the load balancing plug-in module is used for modifying and setting a terminal address of a distributed object storage service in the service registration and authentication node or a terminal address of a server object storage service in the service registration and authentication node according to a balancing rule and monitoring state information of the acquisition monitoring module;
a link is established between the acquisition monitoring module and the storage service gateway proxy node, and the acquisition monitoring module is used for monitoring the state of a storage service gateway proxy node server and feeding the monitored state information back to a load balancing plug-in module of the service registration and authentication node;
the storage service gateway proxy node is a service gateway responsible for object storage service, and is used for establishing connection with a client and storing or downloading data between the storage service gateway proxy node and the storage node;
the storage node is used for storing data;
the balancing rule is that the terminal address in the service registration and authentication node is modified in a polling mode according to the IP address list of the proxy node server of the storage service gateway;
the load balancing plug-in module modifies the service registration and the terminal address of the distributed object storage service in the authentication node in a polling mode by using the IP address list of the proxy node server of the storage service gateway to realize load balancing;
the operation method of the system comprises the following steps:
an authentication link is established between the client and the service registration and authentication node, the client acquires a terminal address of the distributed object storage service through the authentication link, the client establishes a transmission link with the proxy server of the appointed storage service gateway proxy node according to the acquired terminal address, establishes a load balancing link between the authentication nodes and between the storage service gateway proxy nodes, the servers of the proxy nodes of the storage service gateways are balanced, processed, collected and monitored through the load balancing link, the service registration and authentication node modifies and sets the terminal address of the distributed object storage service in the service registration and authentication node or the terminal address of the server object storage service in the service registration and authentication node according to the results of the equalization processing and the acquisition monitoring, and the equalization processing and the acquisition monitoring process are realized through the system.
2. The system for implementing automatic load balancing of openstack swift service according to claim 1, wherein the balancing rule is to set a terminal address of a server object storage service in a service registration and authentication node according to a network bandwidth occupation condition of a storage service gateway proxy node server.
3. The system for realizing automatic load balancing of the openstack swift service according to claim 2, wherein the load balancing plug-in module sets a terminal address of a server object storage service in the service registration and authentication node according to a network bandwidth occupation condition of the storage service gateway proxy node server.
4. The system for realizing automatic load balancing of the openstack swift service according to claim 3, wherein the load balancing plugin module sets the weight and the sequence of each storage service gateway proxy server in the proxy server IP address list according to the bandwidth network state of each storage service gateway proxy server to realize load balancing.
5. The system for implementing openstack swift service automatic load balancing according to claim 1, wherein said collection monitoring module is disposed at a service registration and authentication node.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010619780.3A CN111818147B (en) | 2020-06-30 | 2020-06-30 | System and method for realizing automatic load balancing of openstack swift service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010619780.3A CN111818147B (en) | 2020-06-30 | 2020-06-30 | System and method for realizing automatic load balancing of openstack swift service |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111818147A CN111818147A (en) | 2020-10-23 |
| CN111818147B true CN111818147B (en) | 2022-09-02 |
Family
ID=72856802
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010619780.3A Active CN111818147B (en) | 2020-06-30 | 2020-06-30 | System and method for realizing automatic load balancing of openstack swift service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111818147B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113157441B (en) * | 2021-03-24 | 2023-03-14 | 北京云上曲率科技有限公司 | Distributed decentralized load balancing method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106453564A (en) * | 2016-10-18 | 2017-02-22 | 北京京东尚科信息技术有限公司 | Elastic cloud distributed massive request processing method, device and system |
| CN107341044A (en) * | 2017-07-13 | 2017-11-10 | 郑州云海信息技术有限公司 | A kind of distributive data center unified monitoring framework and method |
| CN109698847A (en) * | 2017-10-23 | 2019-04-30 | 航天信息股份有限公司 | A kind of read method and system of the cloud storage system equally loaded based on the request processing time |
| CN110324194A (en) * | 2019-08-08 | 2019-10-11 | 中国工商银行股份有限公司 | Method, apparatus, system and medium for load balancing |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180329634A1 (en) * | 2017-05-10 | 2018-11-15 | Strato Scale Ltd. | Object overlay for storage-area network (san) appliances |
-
2020
- 2020-06-30 CN CN202010619780.3A patent/CN111818147B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106453564A (en) * | 2016-10-18 | 2017-02-22 | 北京京东尚科信息技术有限公司 | Elastic cloud distributed massive request processing method, device and system |
| CN107341044A (en) * | 2017-07-13 | 2017-11-10 | 郑州云海信息技术有限公司 | A kind of distributive data center unified monitoring framework and method |
| CN109698847A (en) * | 2017-10-23 | 2019-04-30 | 航天信息股份有限公司 | A kind of read method and system of the cloud storage system equally loaded based on the request processing time |
| CN110324194A (en) * | 2019-08-08 | 2019-10-11 | 中国工商银行股份有限公司 | Method, apparatus, system and medium for load balancing |
Non-Patent Citations (1)
| Title |
|---|
| 基于OpenStack架构的视频监控服务平台;曹宁等;《电信技术》;20151225(第12期);51-55 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111818147A (en) | 2020-10-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11641339B2 (en) | Technologies for content delivery network with multi-access edge computing | |
| CN107852604B (en) | System for providing Global Virtual Network (GVN) | |
| US11463511B2 (en) | Model-based load balancing for network data plane | |
| Qi et al. | Assessing container network interface plugins: Functionality, performance, and scalability | |
| CN108200165B (en) | Request Transmission system, method, apparatus and storage medium | |
| US20150003296A1 (en) | System and method for providing p2p based reconfigurable computing and structured data distribution | |
| US20210126965A1 (en) | Systems and methods for scaling down cloud-based servers handling secure connections | |
| CN113572835B (en) | Data processing method, network element equipment and readable storage medium | |
| Hasan et al. | Efficient topology discovery in software defined networks: Revisited | |
| CN112968965B (en) | Metadata service method, server and storage medium for NFV network node | |
| EP4246936A1 (en) | Data processing method, function device and readable storage medium | |
| CN106603352B (en) | System and method for managing load balance of smart home user and application server | |
| US20160380861A1 (en) | Method for ordering monitored packets with tightly-coupled processing elements | |
| CN107689878A (en) | TCP length connection SiteServer LBSs based on name scheduling | |
| CN111585786A (en) | Realization of secret-free building method of big data cluster | |
| CN116633934A (en) | Load balancing method, device, node and storage medium | |
| CN111818147B (en) | System and method for realizing automatic load balancing of openstack swift service | |
| US11700568B2 (en) | Dynamic mapping of nodes responsible for monitoring traffic of an evolved packet core | |
| US11653417B2 (en) | Upgrading access points | |
| US10567262B1 (en) | Dynamic server device monitoring | |
| Manso et al. | Cloud-native SDN controller based on micro-services for transport networks | |
| CN105681266A (en) | Communication cluster method and device for MMTel (MultiMedia Telephony) | |
| CN109308210B (en) | Method for optimizing NFV forwarding service chain performance on multi-core server | |
| CN111880932A (en) | Data storage method and device based on multiple network ports | |
| CN116074160A (en) | Virtual networking public network forwarding method for GPU rendering computing node cluster |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |