CN111818147A - System and method for realizing automatic load balancing of openstack swift service - Google Patents
System and method for realizing automatic load balancing of openstack swift service Download PDFInfo
- Publication number
- CN111818147A CN111818147A CN202010619780.3A CN202010619780A CN111818147A CN 111818147 A CN111818147 A CN 111818147A CN 202010619780 A CN202010619780 A CN 202010619780A CN 111818147 A CN111818147 A CN 111818147A
- Authority
- CN
- China
- Prior art keywords
- service
- node
- load balancing
- authentication
- storage service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/101—Server selection for load balancing based on network conditions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a system for realizing automatic load balancing of openstack swift service, which comprises a client, a service registration and authentication node, a load balancing plug-in module, an acquisition monitoring module, a storage service gateway proxy node and a storage node, wherein the client is connected with the client through a network; the load balancing plug-in module and the acquisition monitoring module form a control link, and the load balancing plug-in module is used for modifying and setting a terminal address of a distributed object storage service in the service registration and authentication node or a terminal address of a server object storage service in the service registration and authentication node according to a balancing rule and monitoring state information of the acquisition monitoring module; the acquisition monitoring module is used for monitoring the state of the proxy node server of the storage service gateway and feeding the monitored state information back to the load balancing plug-in module of the service registration and authentication node. The invention has the advantages that: the openstack swift service automatic load balancing is realized, the equipment utilization efficiency is improved, and the maintenance cost is greatly saved.
Description
Technical Field
The invention relates to the technical field of network communication, in particular to a system and a method for realizing automatic load balancing of openstack swift service.
Background
The Openstack swift provides distributed object storage service, and a service cluster consists of three types of nodes: service registration and authentication node, proxy node, storage node. The service registration and authentication node provides an access entrance of the object storage service and user authentication login, the proxy node is used as a service gateway responsible for the object storage service, and the storage node is responsible for data storage. The client obtains the endpoint of the swift service through the keystone, then establishes connection with the proxy, and uploads data to the storage node through the proxy or downloads data from the storage node to the client. And 2 proxy nodes are configured into a high-availability cluster to realize high availability of the service. This deployment architecture is only suitable for smaller scale application scenarios. For high-throughput scenarios with high-speed, proxy is obviously a bottleneck. Therefore, in a part of production environments, 1 hardware load balancer is deployed in front of a plurality of proxy nodes to meet the needs of a high-concurrency service scene, but the deployment scheme of the hardware load balancer is added in the prior art, but the hardware load balancer can only meet the needs of the high-concurrency service scene to a certain extent, the load balancer itself can become the bottleneck of the performance of the whole storage cluster, and the service capability of the whole storage cluster is not released fundamentally. Meanwhile, due to the addition of the devices, the construction cost and the maintenance cost of the whole storage cluster are increased. The storage service gateway proxy in the prior art cannot realize automatic load balancing, and neither a proxy HA scheme nor a hardware load balancer is added to solve the performance bottleneck problem of the storage gateway fundamentally and in a high cost performance manner.
Disclosure of Invention
Aiming at the problems of load balancing bottleneck and high maintenance cost in the prior art, the invention provides a system and a method for realizing automatic load balancing of openstack swift service.
The invention discloses a system for realizing automatic load balancing of openstack swift service, which comprises a client, a service registration and authentication node, a load balancing plug-in module, an acquisition monitoring module, a storage service gateway proxy node and a storage node, wherein the client is connected with the client through a network;
the client is a terminal used by a user for uploading and downloading data, and the client is used for establishing connection with a proxy node of the storage service gateway and uploading or downloading data;
the service registration and authentication node is a node for authentication login of a user, the user accesses the storage service and performs authentication login through the service registration and authentication node, and the service registration and authentication node is an access entry for accessing the storage service;
the load balancing plug-in module is arranged in the service registration and authentication node, the load balancing plug-in module and the acquisition monitoring module form a control link, and the load balancing plug-in module is used for modifying and setting a terminal address of a distributed object storage service in the service registration and authentication node or a terminal address of a server object storage service in the service registration and authentication node according to a balancing rule and monitoring state information of the acquisition monitoring module;
a link is established between the acquisition monitoring module and the storage service gateway proxy node, and the acquisition monitoring module is used for monitoring the state of a storage service gateway proxy node server and feeding the monitored state information back to a load balancing plug-in module of the service registration and authentication node;
the storage service gateway proxy node is a service gateway responsible for object storage service, and is used for establishing connection with a client and storing or downloading data between the storage service gateway proxy node and the storage node;
the storage nodes are used for storing data.
Further, the balancing rule is to modify the terminal address in the service registration and authentication node in a polling manner according to the IP address list of the proxy node server of the storage service gateway.
Further, the load balancing plug-in module modifies the terminal address of the distributed object storage service in the service registration and authentication node in a polling mode by using the storage service gateway proxy node server IP address list to realize load balancing.
Further, the balancing rule is to set the terminal address of the server object storage service in the service registration and authentication node according to the network bandwidth occupation condition of the storage service gateway proxy node server.
Further, the load balancing plug-in module sets the terminal address of the server object storage service in the service registration and authentication node according to the network bandwidth occupation condition of the storage service gateway proxy node server.
Furthermore, the load balancing plug-in module sets the weight and the sequence of each storage service gateway proxy node server in the proxy server IP address list according to the bandwidth network state of each storage service gateway proxy node server to realize load balancing.
Further, the collection monitoring module is arranged at the service registration and authentication node.
The other purpose of the invention is to provide a method for realizing automatic load balancing of openstack swift service, an authentication link is established between a client and service registration and authentication nodes, the client acquires a terminal address of distributed object storage service through the authentication link, the client establishes a transmission link with a proxy server of a designated storage service gateway proxy node according to the acquired terminal address, a load balancing link is established between the authentication nodes and each storage service gateway proxy node, the servers of each storage service gateway proxy node are balanced, processed, collected and monitored through the load balancing link, the service registration and authentication nodes modify and set the terminal address of the distributed object storage service inside the service registration and authentication nodes or the terminal address of the server object storage service in the service registration and authentication nodes according to the results of the balancing processing and the collected and monitored, the equalization process and acquisition monitoring process are implemented by a system as claimed in claim 1.
Further, the balancing process is configured to modify the terminal address in the service registration and authentication node in a polling manner according to the IP address list of the proxy node server of the storage service gateway.
Further, the balancing process is set to set the terminal address of the server object storage service in the service registration and authentication node according to the network bandwidth occupation condition of the storage service gateway proxy node server.
Compared with the prior art, the invention has the beneficial effects that:
according to the invention, by the scheme of setting the monitoring module and setting the load balancing plug-in the service registration and authentication node, the automatic load balancing of the openstack swift proxy is realized, the utilization efficiency of the equipment is improved, all the storage service gateway proxy node servers are in an effective working state, the overall construction cost and the maintenance cost are effectively reduced, no additional hardware equipment is required, and an HA solution is not introduced.
Drawings
FIG. 1 is a diagram of a system link relationship architecture in accordance with the present invention;
fig. 2 is a functional block diagram of the system of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
The invention is described in further detail below with reference to the attached drawing figures:
referring to fig. 1 and fig. 2, the present embodiment discloses a system for implementing automatic load balancing of openstack swift service, including a client, a service registration and authentication node, a load balancing plug-in module, an acquisition monitoring module, a storage service gateway proxy node, and a storage node.
The Keystone can be used as an authentication system of a service registration and authentication node, the endpoint is a terminal address, and the proxy gateway is a proxy node gateway. In the present embodiment, automatic load balancing of multiple storage gateways proxy is implemented without adding additional hardware devices, and high availability of services is ensured. The system adopts Keystone as an authentication system, a plurality of endpoints are added under the swift service, and one proxy gateway is one endpoint. Keystone certification, although supporting multiple-endpoint, multiple-endpoint cannot achieve load balancing. The end point-based load balancing function is achieved by adding a load balancing plug-in of Keystone, namely plugin in FIG. 1.
The process of load balancing is transparent for the client, that is, the client, i.e., the APP used by the user, acquires the Endpoint of the distributed object storage service through the link 1 in fig. 1, and then connects the designated swift proxy server according to the acquired Endpoint to upload and download data. The load balancing logic process is mainly completed by the cooperative work of the acquisition monitoring and the Keystone load balancing plug-in, two load balancing modes of polling and network states are supported, and the whole system can only work in one load balancing mode.
Polling load balancing, and modifying the Endpoint of the distributed object storage service in the Keystone by using a swift proxy server, namely an IP address list of a storage service gateway proxy node in a polling mode by using a Keystone load balancing plug-in, thereby realizing load balancing. And the acquisition monitoring module continuously monitors the states of all the swift proxy servers through a link a and feeds the state information back to the Keystone plug-in unit through a link b. When a certain swift proxy server is abnormal in service, the Keystone plug-in can remove the abnormal service from the proxy server IP address list so as to ensure the availability of the service.
And based on the load balancing of the network state, setting the endPoint of the server object storage service in the Keystone according to the network bandwidth occupation condition of each swift proxy server so as to achieve the purpose of load balancing. And the acquisition monitoring module continuously monitors the states of all the swift proxy servers through a link a and feeds the state information back to the Keystone load balancing plug-in unit through a link b. And the Keystone load balancing plug-in sets the weight and the sequence of each proxy server in the proxy server IP address list according to the bandwidth network state of each proxy server generation so as to achieve the purpose of load balancing.
Specifically, the client in this embodiment is an APP used by a user and is used for a terminal for uploading and downloading data, and the client is used for establishing a connection with a storage service gateway proxy node and uploading or downloading data; the service registration and authentication node is a node for a user to perform authentication login, the user accesses the storage service and performs authentication login through the service registration and authentication node, the service registration and authentication node is an access entry for accessing the storage service, and in the specific embodiment, the service registration and authentication node can be an authentication system; the load balancing plug-in module is arranged in the service registration and authentication node, the load balancing plug-in module and the acquisition monitoring module form a control link, and the load balancing plug-in module is used for modifying and setting a terminal address of a distributed object storage service in the service registration and authentication node or a terminal address of a server object storage service in the service registration and authentication node according to a balancing rule and monitoring state information of the acquisition monitoring module; a link is established between the acquisition monitoring module and the storage service gateway proxy node, and the acquisition monitoring module is used for monitoring the state of a storage service gateway proxy node server and feeding the monitored state information back to a load balancing plug-in module of the service registration and authentication node; the storage service gateway proxy node is a service gateway responsible for object storage service, and is used for establishing connection with a client and storing or downloading data between the storage service gateway proxy node and the storage node; the storage nodes are used for storage of data, and are specifically storage devices distributed at the respective nodes.
Preferably, the collection monitoring module is disposed at the service registration and authentication node, and may be disposed at other locations in other embodiments of the present invention.
According to the invention, the acquisition monitoring module is arranged to sense the swift proxy running state in real time, so that the load balance and effectiveness are ensured. By setting a keystone load balancing plug-in, the module enables the client to obtain an ideal service URL in an endpoint updating mode, so that a load function is realized. Through the keystone and the operation framework of the acquisition monitoring module, according to the actual operation load of a keystone node, the acquisition monitoring module can be deployed together with the keystone and can also be separately deployed, and the setting is convenient.
By the scheme of setting the monitoring module and setting the load balancing plug-in the service registration and authentication node, automatic load balancing of openstack swift proxy is realized, the utilization efficiency of equipment is improved, all the storage service gateway proxy node servers are in effective working states, the overall construction cost and the maintenance cost are effectively reduced, and an HA solution is not required to be added or introduced.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A system for realizing automatic load balancing of openstack swift service is characterized by comprising a client, a service registration and authentication node, a load balancing plug-in module, an acquisition monitoring module, a storage service gateway proxy node and a storage node;
the client is a terminal used by a user for uploading and downloading data, and the client is used for establishing connection with a proxy node of the storage service gateway and uploading or downloading data;
the service registration and authentication node is a node for authentication login of a user, the user accesses the storage service and performs authentication login through the service registration and authentication node, and the service registration and authentication node is an access entry for accessing the storage service;
the load balancing plug-in module is arranged in the service registration and authentication node, the load balancing plug-in module and the acquisition monitoring module form a control link, and the load balancing plug-in module is used for modifying and setting a terminal address of a distributed object storage service in the service registration and authentication node or a terminal address of a server object storage service in the service registration and authentication node according to a balancing rule and monitoring state information of the acquisition monitoring module;
a link is established between the acquisition monitoring module and the storage service gateway proxy node, and the acquisition monitoring module is used for monitoring the state of a storage service gateway proxy node server and feeding the monitored state information back to a load balancing plug-in module of the service registration and authentication node;
the storage service gateway proxy node is a service gateway responsible for object storage service, and is used for establishing connection with a client and storing or downloading data between the storage service gateway proxy node and the storage node;
the storage nodes are used for storing data.
2. The system for implementing automatic load balancing of openstack swift service according to claim 1, wherein the balancing rule is to modify the terminal address in the service registration and authentication node in a polling manner according to the IP address list of the proxy node server of the storage service gateway.
3. The system for implementing automatic load balancing of openstack swift service as claimed in claim 2, wherein said load balancing plug-in module uses storage service gateway proxy node server IP address list to modify the terminal address of distributed object storage service in service registration and authentication node in polling manner to implement load balancing.
4. The system for implementing automatic load balancing of openstack swift service according to claim 1, wherein the balancing rule is to set a terminal address of a server object storage service in a service registration and authentication node according to a network bandwidth occupation condition of a storage service gateway proxy node server.
5. The system for realizing automatic load balancing of the openstack swift service according to claim 4, wherein the load balancing plug-in module sets the terminal address of the server object storage service in the service registration and authentication node according to the network bandwidth occupation condition of the storage service gateway proxy node server.
6. The system for realizing automatic load balancing of the openstack swift service according to claim 5, wherein the load balancing plugin module sets the weight and the sequence of each storage service gateway proxy server in the proxy server IP address list according to the bandwidth network state of each storage service gateway proxy server to realize load balancing.
7. The system for implementing openstack swift service automatic load balancing according to claim 1, wherein said collection monitoring module is disposed at a service registration and authentication node.
8. A method for realizing automatic load balancing of openstack swift service, it is characterized by that between client end and service register and authentication node an authentication link is set up, the client end can obtain terminal address of distributed object storage service by means of authentication link, and according to the obtained terminal address and proxy server of designated storage service gateway proxy node a transmission link is set up, and between authentication nodes and every storage service gateway proxy node a load balance link is set up, the method comprises the steps that the servers of all the storage service gateway proxy nodes are balanced, collected and monitored through a load balancing link, the service registration and authentication nodes modify and set the terminal addresses of distributed object storage services in the service registration and authentication nodes or the terminal addresses of the server object storage services in the service registration and authentication nodes according to the balancing, collecting and monitoring results, and the balancing, collecting and monitoring processes are achieved through the system in claim 1.
9. The method of claim 8, wherein the balancing process is configured to modify the terminal address in the service registration and authentication node in a polling manner according to the IP address list of the proxy node server of the storage service gateway.
10. The method for implementing automatic load balancing of openstack swift service according to claim 8, wherein the balancing process is configured to set the terminal address of the server object storage service in the service registration and authentication node according to the network bandwidth occupation of the storage service gateway proxy node server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010619780.3A CN111818147B (en) | 2020-06-30 | 2020-06-30 | System and method for realizing automatic load balancing of openstack swift service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010619780.3A CN111818147B (en) | 2020-06-30 | 2020-06-30 | System and method for realizing automatic load balancing of openstack swift service |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111818147A true CN111818147A (en) | 2020-10-23 |
| CN111818147B CN111818147B (en) | 2022-09-02 |
Family
ID=72856802
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010619780.3A Active CN111818147B (en) | 2020-06-30 | 2020-06-30 | System and method for realizing automatic load balancing of openstack swift service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111818147B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113157441A (en) * | 2021-03-24 | 2021-07-23 | 北京云上曲率科技有限公司 | Distributed decentralized load balancing method and system |
| CN115460219A (en) * | 2022-09-19 | 2022-12-09 | 中电云数智科技有限公司 | Distributed file storage gateway system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106453564A (en) * | 2016-10-18 | 2017-02-22 | 北京京东尚科信息技术有限公司 | Elastic cloud distributed massive request processing method, device and system |
| CN107341044A (en) * | 2017-07-13 | 2017-11-10 | 郑州云海信息技术有限公司 | A kind of distributive data center unified monitoring framework and method |
| US20180329634A1 (en) * | 2017-05-10 | 2018-11-15 | Strato Scale Ltd. | Object overlay for storage-area network (san) appliances |
| CN109698847A (en) * | 2017-10-23 | 2019-04-30 | 航天信息股份有限公司 | A kind of read method and system of the cloud storage system equally loaded based on the request processing time |
| CN110324194A (en) * | 2019-08-08 | 2019-10-11 | 中国工商银行股份有限公司 | Method, apparatus, system and medium for load balancing |
-
2020
- 2020-06-30 CN CN202010619780.3A patent/CN111818147B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106453564A (en) * | 2016-10-18 | 2017-02-22 | 北京京东尚科信息技术有限公司 | Elastic cloud distributed massive request processing method, device and system |
| US20180329634A1 (en) * | 2017-05-10 | 2018-11-15 | Strato Scale Ltd. | Object overlay for storage-area network (san) appliances |
| CN107341044A (en) * | 2017-07-13 | 2017-11-10 | 郑州云海信息技术有限公司 | A kind of distributive data center unified monitoring framework and method |
| CN109698847A (en) * | 2017-10-23 | 2019-04-30 | 航天信息股份有限公司 | A kind of read method and system of the cloud storage system equally loaded based on the request processing time |
| CN110324194A (en) * | 2019-08-08 | 2019-10-11 | 中国工商银行股份有限公司 | Method, apparatus, system and medium for load balancing |
Non-Patent Citations (1)
| Title |
|---|
| 曹宁等: "基于OpenStack架构的视频监控服务平台", 《电信技术》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113157441A (en) * | 2021-03-24 | 2021-07-23 | 北京云上曲率科技有限公司 | Distributed decentralized load balancing method and system |
| CN113157441B (en) * | 2021-03-24 | 2023-03-14 | 北京云上曲率科技有限公司 | Distributed decentralized load balancing method and system |
| CN115460219A (en) * | 2022-09-19 | 2022-12-09 | 中电云数智科技有限公司 | Distributed file storage gateway system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111818147B (en) | 2022-09-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11150963B2 (en) | Remote smart NIC-based service acceleration | |
| US11641339B2 (en) | Technologies for content delivery network with multi-access edge computing | |
| CN107852604B (en) | System for providing Global Virtual Network (GVN) | |
| Qi et al. | Assessing container network interface plugins: Functionality, performance, and scalability | |
| US10432535B2 (en) | Performing a specific action on a network packet identified as a message queuing telemetry transport (MQTT) packet | |
| CN107948324B (en) | Request Transmission system, method, apparatus and storage medium | |
| CN108200165B (en) | Request Transmission system, method, apparatus and storage medium | |
| CN110266822B (en) | Shared load balancing implementation method based on nginx | |
| CN106713084B (en) | Multi-gateway system of Internet of things | |
| CN111818147B (en) | System and method for realizing automatic load balancing of openstack swift service | |
| US12101710B2 (en) | Data processing method and apparatus, network element device, storage medium, and program product | |
| CN105119787B (en) | A kind of public internet access system and method based on software definition | |
| Hasan et al. | Efficient topology discovery in software defined networks: Revisited | |
| CN109391661B (en) | Block chain networking method and system for terminal of Internet of things | |
| CN113225214A (en) | Method and device for cooperative management of edge CDN node and computer readable medium | |
| CN112968965B (en) | Metadata service method, server and storage medium for NFV network node | |
| CN113810512A (en) | Internet of things terminal access system, method and device and storage medium | |
| EP4246936A1 (en) | Data processing method, function device and readable storage medium | |
| US20160380861A1 (en) | Method for ordering monitored packets with tightly-coupled processing elements | |
| CN106603352B (en) | System and method for managing load balance of smart home user and application server | |
| CN111182022A (en) | Data transmission method and device, storage medium and electronic device | |
| CN111585786A (en) | Realization of secret-free building method of big data cluster | |
| CN116633934A (en) | Load balancing method, device, node and storage medium | |
| CN110601989A (en) | Network traffic balancing method and device | |
| CN105681266A (en) | Communication cluster method and device for MMTel (MultiMedia Telephony) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |