CN214202379U - Distributed shared file system - Google Patents
Distributed shared file system Download PDFInfo
- Publication number
- CN214202379U CN214202379U CN202022713586.XU CN202022713586U CN214202379U CN 214202379 U CN214202379 U CN 214202379U CN 202022713586 U CN202022713586 U CN 202022713586U CN 214202379 U CN214202379 U CN 214202379U
- Authority
- CN
- China
- Prior art keywords
- file system
- distributed
- load balancer
- storage
- shared file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The utility model discloses a distributed shared file system, the distributed shared file system includes the DNS server that is used for distributing load equalizer for the customer end, is used for dispatching the access request of customer end to the load equalizer of storage container department, is used for handling the storage container of the data that the access request points to and the distributed storage cluster that is used for storing data in the distributed storage cluster; the client is connected with the DNS server through a route, the client and the corresponding load balancer are located in the same subnet, and each storage container and the distributed storage cluster are located in the same network. The utility model provides a technical scheme can improve distributed sharing file system's stability.
Description
Technical Field
The utility model relates to a network architecture technical field, in particular to distributing type shared file system.
Background
With the continuous development of the fields of cloud computing, cloud storage and the like and the explosive growth of data, the performance requirements of users on distributed storage are higher and higher. However, the existing distributed shared file system has poor stability when dealing with large data traffic. There is therefore a need for a more stable distributed shared file system.
SUMMERY OF THE UTILITY MODEL
The application aims to provide a distributed shared file system, which can improve the stability of the distributed shared file system.
To achieve the above object, the present application provides a distributed shared file system, which includes a DNS server for allocating a load balancer to a client, a load balancer for scheduling an access request of the client at a storage container, a storage container for processing data pointed by the access request in a distributed storage cluster, and a distributed storage cluster for storing the data;
the client is connected with the DNS server through a route, the client and the corresponding load balancer are located in the same subnet, and each storage container and the distributed storage cluster are located in the same network.
Further, the storage container is a container realized based on a docker technology, and a ceph distributed file system is deployed in the distributed storage cluster.
Further, a load balancer and at least one storage container constitute a service unit in the distributed shared file system.
Further, each service unit is managed by a daemon process in the distributed shared file system.
Further, the storage containers are in the same management network, and the management network is managed by a daemon process.
Further, the same load balancer is bound to only one subnet.
Further, the load balancer and the storage container may be deployed in the same service device or cluster, or may be deployed in different service devices or clusters.
From top to bottom, the utility model provides a file system is shared to distributing user's data flow to a plurality of storage container and handle through multistage load balancing's mode, when having utilized idle resource, has reduced the too high possibility of certain node load to the holistic stability of system has been improved. Specifically, the DNS server may mount the client to the corresponding load balancer through the first-level load balancing. And the load balancer schedules the flow of the client to the storage container through the second-stage load balancing. The storage container analyzes the access request of the client, and finally, the data can be stored or read in the distributed storage cluster. Therefore, the system architecture with multi-level load balancing is introduced, data flow of a user can be reasonably distributed between the load balancer and the storage container, data abnormity caused by overhigh load of a certain load balancer or a certain storage container is avoided, and the overall stability of the distributed shared file system is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic diagram of a system architecture of a distributed shared file system according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a network configuration of a distributed shared file system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.
Referring to fig. 1, the distributed shared file system provided in the present application may include a DNS server, a load balancer, a storage container (e.g., a container implemented based on a docker technology), and a distributed storage cluster (e.g., a ceph distributed file system), where the load balancer and the storage container may be deployed in the same service device or cluster, or may be deployed in different service devices or clusters, and one load balancer and at least one storage container may form an independent service unit and are connected to each other based on network communication. The storage container is connected with the distributed storage cluster based on network communication.
The distributed shared file system provided by the application provides application service for user accessible operation, a user can access the application service through a client browser or a locally installed application program, a file storage service is created based on the application service, and a file storage server is mounted to the distributed shared file system, so that various data access requests are initiated to the distributed shared file system, and data storage to a distributed storage cluster and read-write operation of stored data are achieved.
Specifically, when a user creates a file storage service through a client, the distributed shared file system can create a mounting point for the file storage service and feed back mounting point information to the client, wherein one mounting point corresponds to at least one service unit, and the mounting point information comprises a uniquely identified domain name set for the file storage service by the distributed shared file system; meanwhile, after the system completes the creation of the service unit, the IP address of the load balancer in the service unit, the domain name of the mounting point and the mapping relation thereof can be configured in the DNS server. When the client executes the mount operation, the client needs to request the DNS server to resolve the mount domain name.
When a client initiates a DNS analysis request aiming at a received mounting domain name, the DNS server can feed back the IP address of the corresponding load balancer to the client based on the mapping relation between the mounting domain name and the IP address of each load balancer, so that the client can establish communication connection with the corresponding load balancer based on the IP address, and mounting of file storage service is realized.
Unlike the ordinary DNS server, the DNS server in the present embodiment may execute a first-level load balancing policy when performing domain name resolution. Specifically, the DNS server may monitor the current load status of each load balancer. The load condition may be characterized by operating parameters on the load balancer. For example, the load status may be characterized by the number of connections of the load balancer, with the greater the number of connections, the greater the load. Of course, in practical applications, the load state may also be characterized by other parameters according to the needs or the difficulty of parameter acquisition. When one mounting domain name corresponds to the IP addresses of a plurality of load balancers, after the DNS server acquires the current load state of each load balancer, a target load balancer with a smaller load can be selected from the load balancers according to the acquired load state, and the IP address of the target load balancer is fed back to the client, so that the client is mounted on the target load balancer.
In one embodiment, when the current load state of the load balancer is characterized by the connection number of the load balancer, the load balancer determining unit in the DNS server may identify the load balancer with the smallest connection number among the load balancers, and use the load balancer with the smallest connection number as the allocated target load balancer. Of course, in practical applications, it is also possible to screen out a plurality of load balancers whose connection number is less than or equal to a certain fixed value, and then randomly select one of the load balancers as a target load balancer.
In a distributed shared file system, the load balancer and storage container may constitute a service unit. In a distributed shared file system, the number of service units created for the same file storage service may be one or more. In a service unit, a load balancer and one or more storage containers may be included, and when a file storage service is mounted to the load balancer, an access request from a client is sent to the load balancer and forwarded by the load balancer to a storage container in the same service unit. The load balancer can execute a second-level load balancing strategy to select one storage container to process the access request of the client. The load balancing policy of the second stage may include selecting a storage container in combination with the weight coefficient and the data traffic currently and actually processed by each storage container.
Specifically, a corresponding weight coefficient may be set in advance for each storage container according to the service performance thereof. The weighting factor may characterize the ability to handle data traffic, with higher weighting factors generally also being higher. Where service performance may be determined based on maximum bandwidth capacity, CPU, and other factors that affect traffic handling capacity. Further, a weight coefficient ratio is obtained according to the weight coefficient of each storage container, for example, a service unit includes 3 storage containers, and the weight coefficient ratio of the three storage containers is 5: 4: 2.
when the load balancer selects the storage container based on the second-level load balancing policy, the weight coefficient proportion of each storage container in the service unit and the data traffic currently being processed by each storage container can be obtained, and a target storage container is determined, wherein the determination principle is that after an access request is dispatched to the target storage container, the traffic proportion of the changed existing data traffic of each storage container can be closer to the coefficient proportion. For example, in the above example, the third storage container may be the target storage container. After a new access request is scheduled to a third storage container, the traffic proportion of the changed existing data traffic may become 10: 7: 3, thereby providing a comparison of 10: 7: 2 is closer to the weight factor ratio 5: 4: 2, therefore, the reasonable distribution of the data flow to each storage container can be ensured, and the condition that part of the storage containers are full and the other storage containers are idle is avoided.
In one embodiment, each service unit may be managed by a daemon process in the system. The daemon process can monitor the state parameters of each storage container in the service unit. In practical applications, the status parameters of the storage container may include information such as an operating status, a bandwidth, and a connection number of the storage container. And the daemon process acquires the state parameters.
Specifically, the operating state of the storage container may be represented by a state code such as active, error, or the like. Wherein active can represent normal work, error can represent abnormal work. Of course, the status code may also characterize an unresponsive, reset, or intermediate status, as desired. Aiming at the storage containers with abnormal work, the daemon process can remove the storage containers with abnormal work from the corresponding service units, and adds a corresponding number of storage containers in the corresponding service units, so that the number of the available storage containers in the service units can be stabilized at a fixed value.
In this embodiment, the Storage container runs an NAS (Network Attached Storage) service, encapsulates the distributed Storage cluster and exposes the encapsulated distributed Storage cluster to a user to provide a Network Storage service, and can analyze a received access request of the client through the NAS service, so as to finally perform processing such as reading, writing, adding, deleting and the like on data in the distributed Storage cluster. The distributed storage cluster serves as a back end of load balancing and provides service capacity with high availability (namely, downtime is reduced, high availability of the service is kept) and load balancing (namely, flow is split, throughput is increased, and data processing capacity is enhanced) in a cluster service mode.
In one embodiment, when the file storage service of the client needs to improve the service performance, a new service unit may be added at a corresponding mount point, and an IP address of a load balancer in the new service unit may be reported to the DNS server. Therefore, when the DNS server executes a first-level load balancing strategy, the client can be mounted to a newly-added load balancer at a chance, the expansion of the distributed shared file system can be rapidly realized through the mode, the requirement for expanding the service scale is met, and meanwhile the similar linear improvement of the service performance of the system can be realized through the linear expansion mode. Similarly, if the service unit at the mount point needs to be reduced, the IP address of the load balancer in the service unit to be reduced can be deleted from the DNS server, so that no new access request is mounted to the load balancer, and when the traffic processing on the storage container is completed, the service unit can be deleted, and the processing resource is released. In this way, high scalability of the system can be achieved.
In an embodiment of the present application, the distributed shared file system further includes a management server for managing and coordinating the DNS server and each service device. The management server is respectively connected with a DNS server and service equipment for deploying a load balancer and a storage container based on network communication, generates a corresponding instruction according to a request of a user for creating a file storage service received by an application service, and sends the corresponding instruction to the service equipment so as to indicate the service equipment to correspondingly create the load balancer and the storage container and simultaneously configure an IP address for the newly created load balancer; the management server can configure mounting information such as a mounting domain name for the file storage service, feed the mounting information back to the user client, and simultaneously send the mounting domain name, the IP address of the load balancer and the corresponding relation of the IP address to the DNS server. When detecting that the file storage service needs to expand or contract, sending a corresponding control instruction to the service equipment to indicate the service equipment to increase or decrease the service units corresponding to the file storage service, and reporting the IP address of a load balancer in the increased or decreased service units to the DNS server to update the records of the mounted domain name and the IP address on the DNS server.
Referring to fig. 2, when a corresponding service unit is created for a file storage service mount point of a client, a distributed shared file system may set a network of a load balancer and a storage container in the service unit.
The load balancer and the corresponding client can be in the same subnet, so that the client can directly access the load balancer after being mounted on the load balancer. Furthermore, the same load balancer can be bound to only one subnet. Through the way of subnet division, the load balancers mounted on the clients of different users can be positioned in different subnets, thereby realizing the isolation among the users.
Each storage container and the distributed storage cluster can be located in the same network, for example, in the same trillion storage, so that rapid data interaction can be performed between the storage containers and the distributed storage cluster. In addition, the storage containers can be in the same management network, and the management network can be managed by a daemon process.
In the distributed shared file system, the DNS server may be in a single network environment, and may be connected to a subnet where the client of the user is located through a route, thereby providing a domain name resolution service for the client of each user.
In practical application, the load balancer can also be kept highly available through a master-slave mode. Each main load balancer can be provided with a backup load balancer, and when the main load balancer cannot provide services, the backup load balancer can be upgraded to the main load balancer and continue to provide services, so that high availability and stability of the system are guaranteed.
From top to bottom, the utility model provides a file system is shared to distributing user's data flow to a plurality of storage container and handle through multistage load balancing's mode, when having utilized idle resource, has reduced the too high possibility of certain node load to the holistic stability of system has been improved. Specifically, the DNS server may mount the client to the corresponding load balancer through the first-level load balancing. And the load balancer schedules the flow of the client to the storage container through the second-stage load balancing. The storage container analyzes the access request of the client, and finally, the data can be stored or read in the distributed storage cluster. Therefore, the system architecture with multi-level load balancing is introduced, data flow of a user can be reasonably distributed between the load balancer and the storage container, data abnormity caused by overhigh load of a certain load balancer or a certain storage container is avoided, and the overall stability of the distributed shared file system is improved.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the present invention, and any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention should be included within the protection scope of the present invention.
Claims (7)
1. A distributed shared file system, comprising a DNS server for allocating a load balancer to clients, a load balancer for scheduling access requests of clients at storage containers, storage containers for processing data pointed by the access requests in the distributed storage clusters, and distributed storage clusters for storing the data;
the client is connected with the DNS server through a route, the client and the corresponding load balancer are located in the same subnet, and each storage container and the distributed storage cluster are located in the same network.
2. The distributed shared file system of claim 1, wherein the storage container is a container implemented based on a docker technology, and wherein a ceph distributed file system is deployed in the distributed storage cluster.
3. The distributed shared file system of claim 1, wherein a load balancer and at least one storage container comprise a service unit in the distributed shared file system.
4. The distributed shared file system of claim 3, wherein each of the service units is managed by a daemon process in the distributed shared file system.
5. The distributed shared file system according to claim 1 or 4, wherein each of the storage containers is in the same management network, and the management network is managed by a daemon process.
6. The distributed shared file system of claim 1, wherein the same load balancer is bound to only one subnet.
7. The distributed shared file system of claim 1, wherein the load balancer and the storage container are deployed in a same service device or cluster, or in different service devices or clusters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202022713586.XU CN214202379U (en) | 2020-11-20 | 2020-11-20 | Distributed shared file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202022713586.XU CN214202379U (en) | 2020-11-20 | 2020-11-20 | Distributed shared file system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN214202379U true CN214202379U (en) | 2021-09-14 |
Family
ID=77646748
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202022713586.XU Active CN214202379U (en) | 2020-11-20 | 2020-11-20 | Distributed shared file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN214202379U (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114900485A (en) * | 2022-05-06 | 2022-08-12 | 阿里巴巴（中国）有限公司 | Method, electronic equipment and system for accessing network file storage |
-
2020
- 2020-11-20 CN CN202022713586.XU patent/CN214202379U/en active Active
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114900485A (en) * | 2022-05-06 | 2022-08-12 | 阿里巴巴（中国）有限公司 | Method, electronic equipment and system for accessing network file storage |
| CN114900485B (en) * | 2022-05-06 | 2024-05-31 | 阿里巴巴（中国）有限公司 | Method, electronic equipment and system for accessing network file storage |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9971823B2 (en) | Dynamic replica failure detection and healing | |
| CN112532675B (en) | Method, device and medium for establishing network edge computing system | |
| US7523454B2 (en) | Apparatus and method for routing a transaction to a partitioned server | |
| KR100745893B1 (en) | Autonomic failover in the context of distributed web services | |
| CN112445774A (en) | Distributed shared file system and data processing method thereof | |
| US9454444B1 (en) | Using location tracking of cluster nodes to avoid single points of failure | |
| JP5582344B2 (en) | Connection management system and connection management server linkage method in thin client system | |
| US7584292B2 (en) | Hierarchical system configuration method and integrated scheduling method to provide multimedia streaming service on two-level double cluster system | |
| US7685312B1 (en) | Resource location by address space allocation | |
| JP2023532947A (en) | Data transfer method, proxy server, storage medium and electronic device | |
| US8578053B2 (en) | NAS load balancing system | |
| US20030110263A1 (en) | Managing storage resources attached to a data network | |
| EP3547102B1 (en) | Object storage system with multi-level hashing function for storage address determination | |
| WO2012118878A1 (en) | Capabilities based routing of virtual data center service request | |
| CN107105013B (en) | File processing method, server, terminal and system | |
| CN113014611B (en) | Load balancing method and related equipment | |
| JP6272190B2 (en) | Computer system, computer, load balancing method and program thereof | |
| WO2003050707A1 (en) | Managing storage resources attached to a data network | |
| KR20070032441A (en) | Load balancing system based on fuzzy grouping and the load balancing method | |
| CN214202379U (en) | Distributed shared file system | |
| CN112532758A (en) | Method, device and medium for establishing network edge computing system | |
| CN109005071B (en) | Decision deployment method and scheduling equipment | |
| US10812390B2 (en) | Intelligent load shedding of traffic based on current load state of target capacity | |
| van Renesse et al. | Autonomic computing: A system-wide perspective | |
| Wei et al. | Towards a cloud storage data management model based on RNPT network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| GR01 | Patent grant | ||
| GR01 | Patent grant |