CN108023953B - High-availability implementation method and device for FTP service - Google Patents
High-availability implementation method and device for FTP service Download PDFInfo
- Publication number
- CN108023953B CN108023953B CN201711261661.XA CN201711261661A CN108023953B CN 108023953 B CN108023953 B CN 108023953B CN 201711261661 A CN201711261661 A CN 201711261661A CN 108023953 B CN108023953 B CN 108023953B
- Authority
- CN
- China
- Prior art keywords
- ftp
- address
- ftp service
- virtual
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000013507 mapping Methods 0.000 claims abstract description 68
- 238000012544 monitoring process Methods 0.000 claims description 57
- 230000004044 response Effects 0.000 claims description 11
- 238000012423 maintenance Methods 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims 2
- 238000010586 diagram Methods 0.000 description 11
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/141—Setup of application sessions
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
Abstract
The application provides a high availability realization method and device of an FTP service, wherein the method comprises the following steps: providing a virtual IP address and establishing a mapping relation between the virtual IP address and physical IP addresses of a plurality of FTP service nodes; receiving a request for establishing an FTP session, which is sent by a client aiming at a virtual IP address; and forwarding the request to a currently available FTP service node according to the mapping relation so as to establish an FTP session between the currently available FTP service node and the client. And an FTP service cluster is constructed to provide FTP service, and the request for establishing the FTP session is scheduled among a plurality of FTP service nodes through a master node, so that the high availability of the FTP service based on the HDFS is realized.
Description
Technical Field
The application relates to an internet data communication technology, in particular to a high-availability realization method and device of an FTP service.
Background
Hadoop is a distributed computing platform that can be easily constructed and used by users. The user can easily develop and run the application program for processing mass data on the Hadoop.
The Hadoop Distributed File System (HDFS) is designed to fit distributed file systems running on general purpose hardware. HDFS is characterized by high fault-tolerance (fault-tolerant) and is designed to be deployed on inexpensive (low-cost) hardware. And it can provide high throughput (high throughput) to access the data of the application program, and is suitable for the application program with the huge data set (largedata set). HDFS can enable streaming access to data in a file system.
The HDFS-over-FTP is an open-source, simple and easy-to-use small tool for realizing the downloading and uploading functions of the HDFS system, can be used as a management tool, and is fast and convenient. But tends to make FTP service unavailable when the number of connections is too large.
Disclosure of Invention
In view of this, embodiments of the present application provide a method and an apparatus for implementing high availability of an FTP service, so as to solve a technical problem that an existing HDFS-over-FTP tool cannot implement high availability.
According to an aspect of an embodiment of the present application, there is provided a method for implementing a high availability of an FTP service, the method including: providing a virtual IP address and establishing a mapping relation between the virtual IP address and physical IP addresses of a plurality of FTP service nodes; receiving a request for establishing the FTP session sent by a client aiming at the virtual IP address; and forwarding the request to a currently available FTP service node according to the mapping relation so as to establish an FTP session between the currently available FTP service node and the client.
In one embodiment, the method for implementing the high availability of the FTP service further comprises: judging whether the FTP service node is available or not through heartbeat information; and in response to the unavailability of one FTP service node, deleting the physical IP address of the unavailable FTP service node from the mapping relation.
In one embodiment, forwarding the request to a currently available FTP service node according to the mapping relationship, so as to establish an FTP session between the currently available FTP service node and the client comprises: carrying out load monitoring on the plurality of FTP service nodes; and forwarding the request to an FTP service node according to the mapping relation and an LVS load balancing rule so as to establish an FTP session between the FTP service node and the client.
In one embodiment, providing a virtual IP address and establishing a mapping relationship between the virtual IP address and physical IP addresses of a plurality of FTP service nodes comprises: providing the same virtual IP address together with a backup server and establishing a mapping relation between the virtual IP address and physical IP addresses of a plurality of FTP service nodes; and sending heartbeat information to the standby server so that the standby server takes over the virtual IP address and continues to provide the FTP service based on the virtual IP address to the client when the heartbeat information is not received.
In one embodiment, the method for implementing the high availability of the FTP service further comprises: receiving a request for acquiring the monitoring data of the FTP service node; and returning corresponding monitoring data according to the monitoring item in the request, wherein the monitoring item comprises one or more of the IP address of the client, the FTP connection quantity and the connection time.
According to another aspect of the embodiments of the present application, there is provided a method for implementing a high availability of an FTP service, the method including: receiving a request for establishing an FTP session, which is transmitted by a client side forwarded by a main node and is aimed at a virtual IP address, wherein the virtual IP address and a local physical IP address establish a mapping relation by the main node; and returning a response message according to the IP address of the client in the request and establishing an FTP session with the client.
In one embodiment, receiving a request from a client forwarded by a master node to establish an FTP session for a virtual IP address further comprises: and during the downtime of the main node, receiving a request for establishing the FTP session, which is sent by a client side forwarded by a standby server of the main node and aims at the virtual IP address.
In one embodiment, the method for implementing the high availability of the FTP service further comprises: and storing the file uploaded by the client to a Hadoop distributed file system, and backing up the file in a preset mirror image directory by the Hadoop distributed file system.
In one embodiment, the method for implementing the high availability of the FTP service further comprises: establishing FTP session with other clients; and responding to the request of other clients for accessing the file, and accessing the mirror image directory to acquire the backup file.
According to another aspect of the embodiments of the present application, there is provided an apparatus for implementing high availability of an FTP service, the apparatus including: the IP address mapping module is used for providing a virtual IP address and establishing a mapping relation between the virtual IP address and the physical IP addresses of the FTP service nodes; a first receiving module, configured to receive a request for establishing an FTP session, where the request is sent by a client for the virtual IP address; and the forwarding module is used for forwarding the request to a currently available FTP service node according to the mapping relation so as to establish an FTP session between the currently available FTP service node and the client.
In one embodiment, the apparatus for implementing high availability of the FTP service further comprises: the node judging module is used for judging whether the FTP service node is available or not through the heartbeat information; and the IP address maintenance module is used for responding to the unavailability of one FTP service node and deleting the physical IP address of the unavailable FTP service node from the mapping relation.
In one embodiment, the forwarding module comprises: the load monitoring unit is used for carrying out load monitoring on the plurality of FTP service nodes; and the forwarding unit is used for forwarding the request to an FTP service node according to the mapping relation and an LVS load balancing rule, and establishing an FTP session between the FTP service node and the client.
In one embodiment, the IP address mapping module includes: the mapping unit is used for providing the same virtual IP address together with the standby server and establishing a mapping relation between the virtual IP address and the physical IP addresses of the FTP service nodes; and the sending unit is used for sending heartbeat information to the standby server so that the standby server takes over the virtual IP address and continues to provide FTP service based on the virtual IP address to the client when the heartbeat information is not received.
In one embodiment, the apparatus for implementing high availability of the FTP service further comprises: the second receiving module is used for receiving a request for acquiring the monitoring data of the FTP service node; and the monitoring data feedback module is used for returning corresponding monitoring data according to the monitoring items in the request, wherein the monitoring items comprise one or more of client IP addresses, FTP connection quantity and connection time.
According to another aspect of the embodiments of the present application, there is provided an apparatus for implementing high availability of an FTP service, the apparatus including: the third receiving module is used for receiving a request for establishing the FTP session, which is sent by the client side forwarded by the main node and is aimed at a virtual IP address, and the virtual IP address and the local physical IP address establish a mapping relation by the main node; and the session establishing module is used for returning a response message according to the IP address of the client in the request and establishing an FTP session with the client.
In one embodiment, the third receiving module is further configured to receive a request for establishing an FTP session, which is sent by a client terminal forwarded by a standby server of the main node for the virtual IP address, during a period in which the main node is down.
In one embodiment, the apparatus for implementing high availability of the FTP service further comprises: and the storage module is used for storing the file uploaded by the client to a Hadoop distributed file system, and the Hadoop distributed file system backs up the file in a preset mirror image directory.
In one embodiment, the apparatus for implementing high availability of the FTP service further comprises: and the mirror directory access module is used for responding to the request of other clients for accessing the file after establishing FTP session with other clients and accessing the mirror directory to acquire the backup file.
According to another aspect of an embodiment of the present application, there is provided an electronic device including a memory and a processor; the memory is used for storing one or more computer instructions, wherein one or more computers are executed by the processor to realize the monitoring method of the data table access amount.
According to another aspect of the embodiments of the present application, there is provided a computer-readable storage medium having stored thereon computer instructions, which when executed by a processor, implement the above method for monitoring the access amount of a data table.
The beneficial effects of the embodiment of the application include: and the master node forwards the request of the client for establishing the FTP session to a currently available FTP service node according to the mapping relation. And an FTP service cluster is constructed to provide FTP service, and the request for establishing the FTP session is scheduled among a plurality of FTP service nodes through a master node, so that the high availability of the FTP service based on the HDFS is realized.
Drawings
The above and other objects, features and advantages of the present application will become more apparent from the following description of embodiments thereof with reference to the accompanying drawings, in which:
FIG. 1 is a schematic diagram of an FTP service system architecture according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an FTP service system architecture according to an embodiment of the present application;
FIG. 3 is a flow chart of a method for implementing high availability of FTP service provided by an embodiment of the present application;
FIG. 4 is a flow chart of a method for implementing high availability of FTP service provided by an embodiment of the present application;
FIG. 5 is a block diagram of a high availability implementation apparatus for FTP service provided by an embodiment of the present application;
FIG. 6 is a block diagram of a high availability implementation apparatus for FTP service provided by an embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.
Detailed Description
The present application is described below based on examples, but the present application is not limited to only these examples. In the following detailed description of the present application, certain specific details are set forth in detail. It will be apparent to one skilled in the art that the present application may be practiced without these specific details. Well-known methods, procedures, components and circuits have not been described in detail so as not to obscure the present application.
Further, those of ordinary skill in the art will appreciate that the drawings provided herein are for illustrative purposes and are not necessarily drawn to scale.
Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise", "comprising", and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is, what is meant is "including, but not limited to".
In the description of the present application, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In addition, in the description of the present application, "a plurality" means two or more unless otherwise specified.
The embodiment of the application establishes an FTP service cluster with a plurality of FTP service nodes based on the HDFS, provides a virtual IP address to a client by a host node and establishes a mapping relation between the virtual IP address and physical IP addresses of the FTP service nodes. And the master node forwards the request of the client for establishing the FTP session to a currently available FTP service node according to the mapping relation. And an FTP service cluster is constructed to provide FTP service, and the request for establishing the FTP session is scheduled among a plurality of FTP service nodes through a master node, so that the high availability of the FTP service based on the HDFS is realized.
Fig. 1 is a schematic structural diagram of an FTP system based on an HDFS according to an embodiment of the present application, including an HDFS10, an FTP service cluster 11, a client 12, a console 13, and an Oracle storage cluster 14; the FTP service cluster 11 includes a master node 111 and a plurality of FTP service nodes 112.
The master node 111 creates a virtual IP address and establishes a mapping relationship between the virtual IP address and physical IP addresses of the plurality of FTP service nodes 112. The master node 111 determines whether each FTP service node 112 is available through heartbeat information and maintains the mapping. If a certain FTP service node 112 is determined to be unavailable (for example, the FTP service node is down or the number of the current FTP session connections exceeds a preset threshold), deleting the physical IP address of the unavailable FTP service node from the mapping relation; and adding the physical IP address of the FTP service node into the mapping relation again when the FTP service node is recovered to the available state.
The FTP service cluster 11 provides the FTP service externally through the virtual IP address, so that the client 12 requests to establish an FTP session through the virtual IP address. The client 12 sends a request for establishing the FTP session to the host node 111 for the virtual IP address, and the host node 111 receives the request and then selects a physical IP address from the mapping relationship according to a preset rule and forwards the physical IP address. The preset rule may be a Linux Virtual Server (Linux Virtual Server) LVS load balancing rule. In order to better realize load balance among the FTP service nodes 112, the FTP service cluster 11 is further configured as an LVS cluster, and the master node 111 performs load monitoring on each FTP service node 112. The LVS cluster adopts an IP load balancing technology and a content-based request distribution technology, wherein a load scheduler has good throughput rate, requests are transferred to different servers to be executed in a balanced mode, and the load scheduler automatically shields the faults of the servers, so that a group of servers form a high-performance and high-availability virtual server. The structure of the whole server cluster is transparent to the client, and the programs of the client and the server do not need to be modified. From the perspective of the LVS cluster, the master node 111 is a load scheduler, the plurality of FTP service nodes 112 are server pools, and the HDFS10 is shared storage of the LVS cluster.
After a physical IP address is selected, the request is forwarded to the corresponding FTP serving node 112. The request contains the client's IP address information and the FTP service node 112 receives the request and returns a reply message according to the protocol to establish an FTP session with the client 12 that sent the request.
In the case where the master node 111 performs real-time load monitoring on the FTP service node 112, the request is forwarded to an FTP service node according to the mapping relationship and the LVS load balancing rules, so as to establish an FTP session between the FTP service node 112 and the client 12. The LVS load balancing rule may perform comprehensive determination according to the monitoring data of all the monitoring items configured by the master node 111. The monitoring items comprise one or more items of client IP addresses, FTP connection quantity, connection time, resource occupancy rate and the like. The master node 111 may periodically or periodically obtain various data from the FTP service node 112 and store the data in a monitoring record (metrics context). At this time, the console 13 requests the master node 111 to acquire monitoring data and requests access to the monitoring record of the master node 111. A corresponding monitoring item (Metrics) is set in the configuration file of the console 13, and corresponding monitoring data is acquired from the monitoring record of the master node 111 according to the setting of the monitoring item. An Oracle storage cluster 14 is arranged at the far end of the console 13 and used for storing the acquired monitoring data, and the acquired monitoring data can be displayed on the console 13 in a report form.
A Keepalived scheme capable of avoiding a single point of failure may also be configured for the FTP service cluster 11. In order to prevent the failure of the master node 111 from rendering the FTP service cluster 11 useless, the master node 111 is provided with one or more backup servers 111', as shown in fig. 2. The main node 111 and the standby server 111' provide the same virtual IP address externally, and the maintained mapping relationship between the virtual IP address and the physical IP address is the same. The main node 111 sends a heartbeat message to the standby server 111 ', and when the standby server 111' does not receive the heartbeat message, it is determined that the main node 111 is down and takes over the virtual IP address. The standby server 111' continues to serve the outside at this time, receives the request of the client 12 for establishing the FTP session for the virtual IP address and forwards the request to the available FTP service node 112.
In the embodiment of the present application, after FTP service node 112 establishes an FTP session with client 12, if client 12 starts to upload a file, FTP service node 112 saves the uploaded file to HDFS10, and if client 12 downloads the file, FTP service node transmits the downloaded file to client 12. Preferably, when the FTP service node 112 stores the uploaded files in the HDFS10, the HDFS10 backs up the uploaded files in a mirror directory to generate corresponding backup files. If other clients 12 request to access the file, the FTP service node 112 determines its mirror directory according to the storage path of the file, and obtains the corresponding backup file from the mirror directory and transmits the backup file to the requesting client 12.
Based on the above description, the present embodiment provides a method for implementing FTP service, which is applicable to the master node 11, and as shown in fig. 3, the method includes the following steps.
S20, providing a virtual IP address and establishing the mapping relation between the virtual IP address and the physical IP addresses of a plurality of FTP service nodes;
s21, receiving a request for establishing FTP session sent by the client aiming at the virtual IP address;
and S22, forwarding the request to a currently available FTP service node according to the mapping relation so as to establish an FTP session between the currently available FTP service node and the client.
The FTP service cluster of the present embodiment has a plurality of FTP service nodes, the host node provides a virtual IP address as an entry of the FTP service cluster, and forwards the request for establishing the FTP session to the currently available FTP service node, thereby realizing high availability of the FTP service based on the HDFS.
In one embodiment, the method for implementing the FTP service in high availability further comprises the following steps.
S23, judging whether the FTP service node is available or not through the heartbeat information;
and S24, in response to the unavailability of one of the FTP service nodes, deleting the physical IP address of the unavailable FTP service node from the mapping relation.
In the embodiment, the mapping relationship is maintained in real time, the physical IP addresses of unavailable FTP service nodes are deleted from the mapping relationship, all the FTP service nodes in the mapping relationship are kept available, and high availability of FTP service is ensured.
In one embodiment, S22 further includes:
s221, carrying out load monitoring on a plurality of FTP service nodes;
s222, the request is forwarded to an FTP service node according to the mapping relation and the LVS load balancing rule, so that an FTP session is established between the FTP service node and the client.
In this embodiment, the FTP service cluster is configured as an LVS cluster, so as to implement load balancing of each FTP service node in the cluster.
In one embodiment, S20 further includes:
s201, providing the same virtual IP address together with a backup server and establishing a mapping relation between the virtual IP address and physical IP addresses of a plurality of FTP service nodes;
s202, sending heartbeat information to the standby server so that the standby server can take over the virtual IP address and continue to provide the FTP service based on the virtual IP address to the client when the standby server cannot receive the heartbeat information.
In one embodiment, the method for implementing the FTP service in high availability further comprises the following steps.
S25, receiving a request for acquiring FTP service node monitoring data;
and S26, returning corresponding monitoring data according to the monitoring item in the request, wherein the monitoring item comprises one or more of the client IP address, the FTP connection quantity and the connection time.
In this embodiment, the host node obtains the monitoring record of the FTP service node, and returns corresponding monitoring data from the monitoring record according to the monitoring item in the request, thereby implementing non-embedded point type acquisition of the monitoring data of the FTP service cluster.
The embodiment of the present application further provides a high availability implementation method of an FTP service, which is applicable to an FTP service node, and as shown in fig. 4, the method includes:
s30, receiving a request for establishing FTP session sent by the client forwarded by the host node aiming at the virtual IP address, wherein the virtual IP address and the local physical IP address are mapped by the host node;
and S31, according to the IP address of the client in the request, returning a response message and establishing an FTP session with the client.
In the embodiment, the FTP service node establishes the FTP session with the client through the FTP session establishment request forwarded by the host node, and the physical IP address of the currently available FTP service node is kept in the mapping relation maintained by the host node, so that the high availability of the FTP service is realized.
In one embodiment, S30 further includes:
and during the downtime of the main node, receiving a request for establishing the FTP session, which is sent by a client side forwarded by a standby server of the main node and aims at the virtual IP address.
The standby server and the main node jointly represent the same virtual IP address, and when the main node is down, the standby server takes over the virtual IP address and provides FTP service to the outside, so that the situation that the FTP service cluster is unavailable due to single-point failure of the main node is prevented.
In one embodiment, the method for implementing the FTP service in high availability further comprises:
and S32, storing the file uploaded by the client into the Hadoop distributed file system, and backing up the file in a preset mirror image directory by the Hadoop distributed file system.
Preferably, the method further comprises:
s33, establishing FTP session with other client;
and S34, responding to the request of other clients for accessing the file, accessing the mirror image directory to acquire the backup file.
The HDFS isolates the storage and the access through the mirror directory, the file uploaded from the FTP service node is backed up in the mirror directory, and if other clients access the file from the FTP service node, the backup file in the mirror directory is accessed, so that the safety of the originally uploaded data is ensured.
Corresponding to the above-mentioned high-availability implementation method of the FTP service, an embodiment of the present application further provides a high-availability implementation apparatus of the FTP service, as shown in fig. 5, the apparatus includes:
the IP address mapping module 40 is used for providing a virtual IP address and establishing a mapping relation between the virtual IP address and the physical IP addresses of a plurality of FTP service nodes;
a first receiving module 41, configured to receive a request sent by a client for establishing an FTP session in response to a virtual IP address;
and the forwarding module 42 is configured to forward the request to a currently available FTP service node according to the mapping relationship, so as to establish an FTP session between the currently available FTP service node and the client.
In one embodiment, the apparatus for implementing high availability of the FTP service further comprises:
the node judging module is used for judging whether the FTP service node is available or not through the heartbeat information;
and the IP address maintenance module is used for responding to the unavailability of one FTP service node and deleting the physical IP address of the unavailable FTP service node from the mapping relation.
In one embodiment, the forwarding module 42 further comprises:
the load monitoring unit is used for carrying out load monitoring on the plurality of FTP service nodes;
and the forwarding unit is used for forwarding the request to an FTP service node according to the mapping relation and the LVS load balancing rule, and establishing an FTP session between the FTP service node and the client.
In one embodiment, the IP address mapping module 40 further comprises:
the mapping unit is used for providing the same virtual IP address together with the standby server and establishing a mapping relation between the virtual IP address and the physical IP addresses of the FTP service nodes;
and the sending unit is used for sending the heartbeat information to the standby server so that the standby server can take over the virtual IP address and continue to provide the FTP service based on the virtual IP address to the client when the heartbeat information is not received.
In one embodiment, the apparatus for implementing high availability of the FTP service further comprises:
the second receiving module is used for receiving a request for acquiring the monitoring data of the FTP service node;
and the monitoring data feedback module is used for returning corresponding monitoring data according to the monitoring items in the request, wherein the monitoring items comprise one or more of the IP address of the client, the FTP connection quantity and the connection time.
Corresponding to the above-mentioned high-availability implementation method of the FTP service, an embodiment of the present application further provides a high-availability implementation apparatus of the FTP service, as shown in fig. 6, the apparatus includes:
a third receiving module 50, configured to receive a request sent by the client to establish an FTP session and forwarded by the host node, where the virtual IP address and the local physical IP address are mapped by the host node,
and the session establishing module 51 is configured to return a response message according to the IP address of the client in the request and establish an FTP session with the client.
In one embodiment, the third receiving module 50 is further configured to receive a request for establishing an FTP session from a client forwarded by a standby server of the main node for a virtual IP address during a period when the main node is down.
In one embodiment, the apparatus for implementing high availability of the FTP service further comprises:
and the storage module is used for storing the file uploaded by the client to the Hadoop distributed file system, and the Hadoop distributed file system backs up the file in a preset mirror image directory.
In one embodiment, the apparatus for implementing high availability of the FTP service further comprises:
and the mirror directory access module is used for responding to the requests of other clients for accessing the files after the FTP session is established with other clients, and accessing the mirror directory to acquire the backup files.
Furthermore, the highly available implementation means of the FTP service may also implement the above program steps through a hardware processor (hardware processor). Referring now to FIG. 7, shown is a block diagram of a computer system 600 suitable for use in implementing the electronic device of an embodiment of the present application. The electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 7, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 606 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the method of the present application when executed by a Central Processing Unit (CPU) 601. It should be noted that the computer readable medium described herein can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present application may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor comprises an IP address mapping module, a first receiving module and a forwarding module; or the processor comprises a third receiving module and a session establishing module.
As another aspect, the present application also provides a non-volatile computer storage medium, which may be the non-volatile computer storage medium included in the apparatus in the above-described embodiments; or it may be a non-volatile computer storage medium that exists separately and is not built into the electronic device. The non-volatile computer storage medium stores one or more programs that, when executed by a device, cause the device to: providing a virtual IP address and establishing a mapping relation between the virtual IP address and physical IP addresses of a plurality of FTP service nodes; receiving a request for establishing the FTP session sent by a client aiming at the virtual IP address; forwarding the request to a currently available FTP service node according to the mapping relation so as to establish an FTP session between the currently available FTP service node and a client; or, causing the apparatus to: receiving a request for establishing an FTP session, which is transmitted by a client side forwarded by a main node and is aimed at a virtual IP address, wherein the virtual IP address and a local physical IP address establish a mapping relation by the main node; and returning a response message according to the IP address of the client in the request and establishing an FTP session with the client.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (20)
1. A method for highly available implementation of FTP services, the method comprising:
providing a virtual IP address and establishing a mapping relation between the virtual IP address and physical IP addresses of a plurality of FTP service nodes, wherein the plurality of FTP service nodes are connected with a distributed file system (HDFS), and the distributed file system (HDFS) is used for storing files;
receiving a request for establishing the FTP session sent by a client aiming at the virtual IP address;
and forwarding the request to a currently available FTP service node according to the mapping relation so as to establish an FTP session between the currently available FTP service node and the client.
2. The method of claim 1, further comprising:
judging whether the FTP service node is available or not through heartbeat information;
and in response to the unavailability of one FTP service node, deleting the physical IP address of the unavailable FTP service node from the mapping relation.
3. The method of claim 1, wherein forwarding the request to a currently available FTP service node according to the mapping relationship, so as to establish an FTP session between the currently available FTP service node and the client comprises:
carrying out load monitoring on the plurality of FTP service nodes;
and forwarding the request to an FTP service node according to the mapping relation and an LVS load balancing rule so as to establish an FTP session between the FTP service node and the client.
4. The method of claim 1, wherein providing a virtual IP address and establishing a mapping between the virtual IP address and physical IP addresses of a plurality of FTP service nodes comprises:
providing the same virtual IP address together with a standby server and establishing a mapping relation between the virtual IP address and physical IP addresses of a plurality of FTP service nodes;
and sending heartbeat information to the standby server so that the standby server takes over the virtual IP address and continues to provide the FTP service based on the virtual IP address to the client when the heartbeat information is not received.
5. The method of claim 1, further comprising:
receiving a request for acquiring the monitoring data of the FTP service node;
and returning corresponding monitoring data according to the monitoring item in the request, wherein the monitoring item comprises one or more of the IP address of the client, the FTP connection quantity and the connection time.
6. A high-availability realization method of FTP service is applied to an FTP service node, and is characterized in that the method comprises the following steps:
receiving a request for establishing an FTP session, which is transmitted by a client side forwarded by a main node and is aimed at a virtual IP address, wherein the virtual IP address and a local physical IP address establish a mapping relation by the main node;
returning a response message according to the IP address of the client in the request and establishing an FTP session with the client;
the FTP service node is connected with a distributed file system (HDFS), and the distributed file system (HDFS) is used for storing files.
7. The method of claim 6, wherein receiving a client-initiated request to establish an FTP session for a virtual IP address forwarded by a host node further comprises:
and during the downtime of the main node, receiving a request for establishing the FTP session, which is sent by a client side forwarded by a standby server of the main node and aims at the virtual IP address.
8. The method of claim 6, further comprising:
and storing the file uploaded by the client to a Hadoop distributed file system, and backing up the file in a preset mirror image directory by the Hadoop distributed file system.
9. The method of claim 8, further comprising:
establishing FTP session with other clients;
and responding to the request of other clients for accessing the file, and accessing the mirror image directory to acquire the backup file.
10. An apparatus for high availability implementation of FTP service, the apparatus comprising:
the system comprises an IP address mapping module, a file transfer module and a file transfer module, wherein the IP address mapping module is used for providing a virtual IP address and establishing a mapping relation between the virtual IP address and physical IP addresses of a plurality of FTP service nodes, the plurality of FTP service nodes are connected with a distributed file system (HDFS), and the distributed file system (HDFS) is used for storing files;
a first receiving module, configured to receive a request for establishing an FTP session, where the request is sent by a client for the virtual IP address;
and the forwarding module is used for forwarding the request to a currently available FTP service node according to the mapping relation so as to establish an FTP session between the currently available FTP service node and the client.
11. The apparatus of claim 10, further comprising:
the node judging module is used for judging whether the FTP service node is available or not through the heartbeat information;
and the IP address maintenance module is used for responding to the unavailability of one FTP service node and deleting the physical IP address of the unavailable FTP service node from the mapping relation.
12. The apparatus of claim 10, wherein the forwarding module comprises:
the load monitoring unit is used for carrying out load monitoring on the plurality of FTP service nodes;
and the forwarding unit is used for forwarding the request to an FTP service node according to the mapping relation and an LVS load balancing rule, and establishing an FTP session between the FTP service node and the client.
13. The apparatus of claim 10, wherein the IP address mapping module comprises:
the mapping unit is used for providing the same virtual IP address together with the standby server and establishing a mapping relation between the virtual IP address and the physical IP addresses of the FTP service nodes;
and the sending unit is used for sending heartbeat information to the standby server so that the standby server takes over the virtual IP address and continues to provide the FTP service based on the virtual IP address to the client when the heartbeat information is not received.
14. The apparatus of claim 10, further comprising:
the second receiving module is used for receiving a request for acquiring the monitoring data of the FTP service node;
and the monitoring data feedback module is used for returning corresponding monitoring data according to the monitoring items in the request, wherein the monitoring items comprise one or more of client IP addresses, FTP connection quantity and connection time.
15. An apparatus for implementing high availability of an FTP service, applied to an FTP service node, the apparatus comprising:
the third receiving module is used for receiving a request for establishing the FTP session, which is sent by the client side forwarded by the main node and is aimed at a virtual IP address, and the virtual IP address and the local physical IP address establish a mapping relation by the main node;
the session establishing module is used for returning a response message according to the IP address of the client in the request and establishing an FTP session with the client;
the FTP service node is connected with a distributed file system (HDFS), and the distributed file system (HDFS) is used for storing files.
16. The apparatus of claim 15, wherein the third receiving module is further configured to receive a request from a client forwarded by a backup server of the master node to establish an FTP session for the virtual IP address during a period when the master node is down.
17. The apparatus of claim 15, further comprising:
and the storage module is used for storing the file uploaded by the client to a Hadoop distributed file system, and the Hadoop distributed file system backs up the file in a preset mirror image directory.
18. The apparatus of claim 17, further comprising:
and the mirror directory access module is used for responding to the request of other clients for accessing the file after establishing FTP session with other clients and accessing the mirror directory to acquire the backup file.
19. An electronic device comprising a memory and a processor; the memory is configured to store one or more computer instructions, wherein the one or more computers are executed by the processor to implement the steps of the method of any one of claims 1 to 5 or any one of claims 6 to 9.
20. A computer readable storage medium having stored thereon computer instructions, characterized in that the instructions, when executed by a processor, carry out the steps of the method of any one of claims 1 to 5 or of any one of claims 6 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711261661.XA CN108023953B (en) | 2017-12-04 | 2017-12-04 | High-availability implementation method and device for FTP service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711261661.XA CN108023953B (en) | 2017-12-04 | 2017-12-04 | High-availability implementation method and device for FTP service |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108023953A CN108023953A (en) | 2018-05-11 |
| CN108023953B true CN108023953B (en) | 2021-07-09 |
Family
ID=62078413
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711261661.XA Expired - Fee Related CN108023953B (en) | 2017-12-04 | 2017-12-04 | High-availability implementation method and device for FTP service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108023953B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109271106B (en) * | 2018-08-31 | 2021-03-05 | 华为技术有限公司 | Message storage method, message reading method, message storage device, message reading device, server and storage medium |
| CN111225003B (en) * | 2018-11-23 | 2022-12-27 | 京东科技控股股份有限公司 | NFS node configuration method and device |
| CN109992531B (en) * | 2019-04-15 | 2020-11-10 | 成都四方伟业软件股份有限公司 | Data storage method and device |
| CN110266761B (en) * | 2019-05-17 | 2022-04-19 | 平安科技（深圳）有限公司 | Load balancing application creation method and device, computer equipment and storage medium |
| CN111131492A (en) * | 2019-12-31 | 2020-05-08 | 中国联合网络通信集团有限公司 | Node access method and system |
| CN111431980B (en) * | 2020-03-17 | 2021-12-17 | 星辰天合（北京）数据科技有限公司 | Distributed storage system and path switching method thereof |
| CN113722084A (en) * | 2020-05-25 | 2021-11-30 | 中移(苏州)软件技术有限公司 | Data processing method and device, electronic equipment and computer storage medium |
| CN113238893B (en) * | 2021-05-11 | 2024-07-26 | 鸬鹚科技(深圳)有限公司 | Disaster recovery system, method, computer equipment and medium for multiple data centers |
| CN113824812B (en) * | 2021-08-27 | 2023-02-28 | 济南浪潮数据技术有限公司 | Method, device and storage medium for HDFS service to acquire service node IP |
| CN115051982B (en) * | 2022-06-14 | 2023-12-01 | 北京天融信网络安全技术有限公司 | Information processing method, system and storage medium based on FTP protocol |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1468002A (en) * | 2002-06-27 | 2004-01-14 | 上海汉唐科技有限公司 | Flow media compression, transmission and storage system based on internet |
| CN1595906A (en) * | 2003-12-17 | 2005-03-16 | 浪潮电子信息产业股份有限公司 | Single address traffic distributor of cluster network |
| CN1649326A (en) * | 2004-12-09 | 2005-08-03 | 武汉大学 | Front end system constructing method of multiple distributor of colony server |
| CN105357280A (en) * | 2015-10-19 | 2016-02-24 | 福建新大陆软件工程有限公司 | Hadoop distributed file system (HDFS) based file tracing file transfer protocol (FTP) system |
| CN106789324A (en) * | 2017-01-09 | 2017-05-31 | 上海轻维软件有限公司 | FTP distributed acquisition methods based on MapReduce |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5793763A (en) * | 1995-11-03 | 1998-08-11 | Cisco Technology, Inc. | Security system for network address translation systems |
| US7039008B1 (en) * | 1997-05-02 | 2006-05-02 | Cisco Technology, Inc. | Method and apparatus for maintaining connection state between a connection manager and a failover device |
| US6944785B2 (en) * | 2001-07-23 | 2005-09-13 | Network Appliance, Inc. | High-availability cluster virtual server system |
| US7756984B2 (en) * | 2004-09-27 | 2010-07-13 | Citrix Systems, Inc. | Systems and methods for virtual host name roaming |
| CN100493086C (en) * | 2005-04-20 | 2009-05-27 | 大唐移动通信设备有限公司 | Method for realizing virtual IP protocol |
| CN101325580B (en) * | 2007-06-15 | 2012-01-25 | 上海亿人通信终端有限公司 | Method for implementing FTP application-layer gateway based on NAT-PT |
| CN102148839B (en) * | 2010-02-09 | 2015-06-03 | 蓝盾信息安全技术股份有限公司 | Method, device and system for managing FTP (file transfer protocol) server |
| CN103475732A (en) * | 2013-09-25 | 2013-12-25 | 浪潮电子信息产业股份有限公司 | Distributed file system data volume deployment method based on virtual address pool |
| CN104320401B (en) * | 2014-10-31 | 2018-06-22 | 北京思特奇信息技术股份有限公司 | A kind of big data storage based on distributed file system accesses system and method |
| CN105337780B (en) * | 2015-12-01 | 2018-09-18 | 迈普通信技术股份有限公司 | A kind of server node configuration method and physical node |
| CN105912677A (en) * | 2016-04-13 | 2016-08-31 | 北京思特奇信息技术股份有限公司 | Method and system for implementing document transmission between FTP and HDFS based on OOZIE |
| CN105744004B (en) * | 2016-04-29 | 2019-08-16 | 凯通科技股份有限公司 | A kind of transmission method and system of the distributed data library file based on FTP |
| CN106372256A (en) * | 2016-09-30 | 2017-02-01 | 浙江大学 | Distributed storage method for massive Argo data |
-
2017
- 2017-12-04 CN CN201711261661.XA patent/CN108023953B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1468002A (en) * | 2002-06-27 | 2004-01-14 | 上海汉唐科技有限公司 | Flow media compression, transmission and storage system based on internet |
| CN1595906A (en) * | 2003-12-17 | 2005-03-16 | 浪潮电子信息产业股份有限公司 | Single address traffic distributor of cluster network |
| CN1649326A (en) * | 2004-12-09 | 2005-08-03 | 武汉大学 | Front end system constructing method of multiple distributor of colony server |
| CN105357280A (en) * | 2015-10-19 | 2016-02-24 | 福建新大陆软件工程有限公司 | Hadoop distributed file system (HDFS) based file tracing file transfer protocol (FTP) system |
| CN106789324A (en) * | 2017-01-09 | 2017-05-31 | 上海轻维软件有限公司 | FTP distributed acquisition methods based on MapReduce |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108023953A (en) | 2018-05-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108023953B (en) | High-availability implementation method and device for FTP service | |
| CN110417842B (en) | Fault processing method and device for gateway server | |
| CN108737270B (en) | Resource management method and device for server cluster | |
| US11157457B2 (en) | File management in thin provisioning storage environments | |
| US9742651B2 (en) | Client-side fault tolerance in a publish-subscribe system | |
| US10313452B2 (en) | Migrating a chat message service provided by a chat server to a new chat server | |
| US10241876B1 (en) | Cooperative fault tolerance and load balancing | |
| CN110703980B (en) | File transmission method and device | |
| WO2017107463A1 (en) | Method and apparatus for delivering cdn addresses | |
| US20170153909A1 (en) | Methods and Devices for Acquiring Data Using Virtual Machine and Host Machine | |
| US11706298B2 (en) | Multichannel virtual internet protocol address affinity | |
| CN111770176B (en) | Traffic scheduling method and device | |
| CN104125294B (en) | A kind of large data safety control method and system | |
| CN109428926B (en) | Method and device for scheduling task nodes | |
| CN112600761A (en) | Resource allocation method, device and storage medium | |
| CN107181774B (en) | Data movement between distributed data centers | |
| CN111427551A (en) | User code operation method of programming platform, equipment and storage medium | |
| CN108881379A (en) | The synchronous method and apparatus of data between a kind of server cluster | |
| CN107818027B (en) | Method and device for switching main name node and standby name node and distributed system | |
| CN114296953A (en) | Multi-cloud heterogeneous system and task processing method | |
| CN110113176B (en) | Information synchronization method and device for configuration server | |
| CN111831503A (en) | Monitoring method based on monitoring agent and monitoring agent device | |
| US9544371B1 (en) | Method to discover multiple paths to disk devices cluster wide | |
| KR101379105B1 (en) | System and method for offering cloud computing service | |
| CN109981778B (en) | Method, device, equipment and storage medium for realizing service of content distribution network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 202, 2 floors, 1-3 floors, No. 11 Shangdi Information Road, Haidian District, Beijing 100085 Applicant after: Beijing Xingxuan Technology Co.,Ltd. Address before: Room 202, 2 floors, 1-3 floors, No. 11 Shangdi Information Road, Haidian District, Beijing 100085 Applicant before: Beijing Xiaodu Information Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210709 |