CN106502828A - A kind of remote copy method based on LVM of optimization - Google Patents
A kind of remote copy method based on LVM of optimization Download PDFInfo
- Publication number
- CN106502828A CN106502828A CN201610913666.5A CN201610913666A CN106502828A CN 106502828 A CN106502828 A CN 106502828A CN 201610913666 A CN201610913666 A CN 201610913666A CN 106502828 A CN106502828 A CN 106502828A
- Authority
- CN
- China
- Prior art keywords
- lvm
- remote
- modules
- request
- tcp
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention be more particularly directed to a kind of remote copy method based on LVM of optimization.The remote copy method based on LVM of the optimization, includes LVM I/O Analysis modules, three part of LVM I/O Process modules and LVM Remote Replication modules based on LVM remote copy frameworks；The LVM I/O Analysis modules are optimized to the I/O request analysis of upper layer application；The I/O handling processes of the LVM I/O Process modules more overwrite request；The LVM Remote Replication modules are modified to NBD modules, are allowed to dock with the inner core of LVM, are realized the long-range transmission of data；LVM Remote Replication modules also have been also carried out optimizing to the parameter of TCP/IP networks, are allowed to more effectively to suitable for remote copy scene.The remote copy method based on LVM of the optimization, can be used in the remote disaster tolerance system of field of cloud calculation, special physical environment is not needed, only original IP network need to be correspondingly improved, remote data is copied to another remote equipment in the method for complete software, stability of the data in remote copy, security and efficiency of transmission is improved.
Description
Technical field
The present invention relates to cloud computing technical field of memory, the remote copy method based on LVM of more particularly to a kind of optimization.
Background technology
LVM（Logical Volume Manager, logical volume management）It is that a kind of dynamic management storage sets in Linux system
Standby administrative mechanism, it pass through bottom physical equipment abstract package, provide to upper strata and continuously deposit with conventional bar storage identical
Storage space.LVM provides complete User space and kernel state administrative mechanism, while the Dynamic date migration provided in LVM technology,
The characteristics such as mirror, snapshot, can provide complete framework for data copy system.
NBD（Network Block Device, network control techology）By providing client and server services, will be long-range
The disk space that main frame is provided is used as block device local.NBD is a kernel module, and it is connect by being defined on kernel
Mouthful, can be with other block device managers in Linux system（Such as LVM）Slitless connection.
Remote copy technology is exactly by the data at source device end, by long-range Connection-copy to one or more target devices
End technology.The technology is usually used in disaster tolerance system, it by using geographic dispersiveness, when a regional loss of data
Afterwards, can be by the fast quick-recovery of the data of distal end.
Above-mentioned situation is based on, the present invention proposes a kind of remote copy method based on LVM of optimization.
Content of the invention
The present invention is in order to make up the defect of prior art, there is provided a kind of simple efficient optimize based on the long-range of LVM
Clone method.
The present invention is achieved through the following technical solutions：
A kind of remote copy method based on LVM of optimization, it is characterised in that：LVM I/O are included based on LVM remote copy frameworks
Analysis modules, three part of LVM I/O Process modules and LVM Remote Replication modules；The LVM I/
O Analysis modules are optimized to the I/O request analysis of upper layer application, for the direct access local data of read request, for
Write request is then sent to local and remote equipment processing module；The LVM I/O Process modules to LVM in Mirror connect
Mouth is optimized, and data are distinguished by the I/O handling processes of more overwrite request, the method for being updated the data by first log again
It is sent to local and remote equipment；The LVM Remote Replication modules are modified to NBD modules, be allowed to
The inner core docking of LVM, realizes the long-range transmission of data；Simultaneously as NBD messages are transmitted via TCP/IP, described
LVM Remote Replication modules also have been also carried out optimizing to the parameter of TCP/IP networks, in adjustment TCP/IP networks
Tcp_sack, tcp_timestamps, tcp_low_latency, tcp_westwood and tcp_bic parameter, is allowed to more effective
To suitable for remote copy scene.
The I/O requests that the LVM I/O Analysis modules are responsible for initiating upper layer application are filtered, by changing lvm_
The request of make_request_fn () function pair I/O is classified, and read request or the request unrelated with duplication directly transfer to LVM certainly
Body mechanism is read from disk, and write request then calls the interface of LVM I/O Process modules to be processed；The LVM I/O
Process modules are responsible for the flow process for building I/O, while docking with Remote Replication modules；The LVM Remote
Replication modules are responsible for the data consistency definition of remote copy and the remote transmission of data；I/O requests are passed through in main side
After by LVM Remote Replication resume modules, it is sent to remote equipment end and successfully submits to daily record to write data disks
Afterwards, update local log information and submit to LVM I/O Process modules；When in LVM I/O Process modules
After mirror receives local and long-range return message, return to application layer and write as work(.
I/O requests recorded daily record by the I/O handling processes of the LVM I/O Process modules to write request first
In information, BIOS is replicated two parts then, portion is sent to local data disks, and another is sent in request queue, finally
Remote Replication module interfaces are called, and opposite end are transmitted data to using multiple threads；When receive local and
After the completion of two requests of remote equipment, request is returned to upper layer application and processed successfully.
The LVM Remote Replication modules are referred to using in real time to the data consistency definition of remote copy
Synchronous mode, i.e., multiple write requests that upper layer application sends have to wait for main side and the data at standby end are simultaneously written and can continue
Execute；When request arrives, first daily record is submitted to according to request order, write data disks again, when writing data disks, press after writing daily record success
Disk is write successively according to order, after only writing daily record and data disks being write while successfully submitting to, just submits sequential update daily record to according to request
Information, finally notifies upper layer request to complete；
The LVM Remote Replication modules are completed based on NBD modules to the remote transmission of data, right first
Main side and standby end are initialized by nbd_init functions and are connected, then using do_nbd_request functions from request queue
Read corresponding ask, using message structure in handle domains load and be sent to standby end；Call nbd_do_it circular waits simultaneously
Call nbd_read_stat to read for held and write plate-like state；After the completion of read-write requests, then from request queue, removal request is simultaneously
Return successfully to upper strata.
The LVM Remote Replication modules include utilizing tcp_ to the optimization of the parameter of TCP/IP networks
Sack parameters have enabled selection answer-mode, lift the reply of out of order message, only send the message that loses；Using tcp_
Timestamps parameters enable TCP timestamps, and more precise control message is retransmitted；Close tcp_low_latency parameters and selection
Tcp_westwood parameters are optimized to the whole utilization situation of bandwidth by throughput evaluation；Biography for quick long range
The defeated Binary Increase Congestion for enabling tcp_bic parameters, more preferably using the GB step velocitys of transmission network.
The invention has the beneficial effects as follows：The remote copy method based on LVM of the optimization, can be used for field of cloud calculation
In remote disaster tolerance system, it is not necessary to special physical environment, only original IP network need to be correspondingly improved, with completely soft
Remote data is copied to another remote equipment by the method for part, improves stability of the data in remote copy, security
And efficiency of transmission.
Description of the drawings
Accompanying drawing 1 is based on LVM remote copy architectural configurations schematic diagrames for the present invention.
Accompanying drawing 2 is LVM I/O Process resume module schematic flow sheets of the present invention.
Accompanying drawing 3 is LVM Remote Replication module I O process optimization schematic diagrames of the present invention.
Accompanying drawing 4 is LVM Remote Replication resume module schematic flow sheets of the present invention.
Accompanying drawing 5 is NBD submodule message structure schematic diagrames in LVM Remote Replication modules of the present invention.
Specific embodiment
In order that the technical problem to be solved, technical scheme and beneficial effect become more apparent, below tie
Drawings and Examples are closed, the present invention will be described in detail.It should be noted that specific embodiment described herein is only used
To explain the present invention, it is not intended to limit the present invention.
The remote copy method based on LVM of the optimization, includes LVM I/O Analysis based on LVM remote copy frameworks
Module, three part of LVM I/O Process modules and LVM Remote Replication modules；The LVM I/O
Analysis modules are optimized to the I/O request analysis of upper layer application, for the direct access local data of read request, for writing
Request is then sent to local and remote equipment processing module；The LVM I/O Process modules to LVM in Mirror interfaces
It is optimized, data are sent out by the I/O handling processes of more overwrite request, the method for being updated the data by first log again respectively
It is sent to local and remote equipment；The LVM Remote Replication modules are modified to NBD modules, are allowed to and LVM
Inner core docking, realize the long-range transmission of data；Simultaneously as NBD messages are transmitted via TCP/IP, the LVM
Remote Replication modules also have been also carried out optimizing to the parameter of TCP/IP networks, adjust the tcp_ in TCP/IP networks
Sack, tcp_timestamps, tcp_low_latency, tcp_westwood and tcp_bic parameter, is allowed to more effectively arrive and fits
For remote copy scene.
The I/O requests that the LVM I/O Analysis modules are responsible for initiating upper layer application are filtered, by changing lvm_
The request of make_request_fn () function pair I/O is classified, and the major function of lvm_make_request_fn () function is
To the Target Device by Mapped Device by address of cache to bottom, and the address of mapping is returned to system,
Read request or the request unrelated with duplication directly transfer to LVM self mechanisms to read from disk, and write request then calls LVM I/O
The interface of Process modules is processed；The LVM I/O Process modules are responsible for the flow process for building I/O, while with
Remote Replication modules are docked, and whole LVM I/O Process modules are that the mirror characteristics change based on LVM is real
Existing, mirror characteristics are that I/O requests are separately sent in local and copy volume；The LVM Remote Replication
Module is responsible for the data consistency definition of remote copy and the remote transmission of data；By I/O requests in main side via LVM
After Remote Replication resume modules, after being sent to remote equipment end and successfully submitting to daily record to write data disks, update
Local log information simultaneously submits to LVM I/O Process modules；When the mirror in LVM I/O Process modules is received
After local and long-range return message, return to application layer and write as work(.
I/O requests recorded daily record by the I/O handling processes of the LVM I/O Process modules to write request first
In information, BIOS is replicated two parts then, portion is sent to local data disks, and another is sent in request queue, finally
Remote Replication module interfaces are called, and opposite end are transmitted data to using multiple threads；When receive local and
After the completion of two requests of remote equipment, request is returned to upper layer application and processed successfully.
Request queue is the buffering asked by I/O.Introduce the mechanism and be because that the I/O of remote copy needs to be sent to remotely
Transmission speed on equipment is differed greatly with locally stored speed, in order to lift the performance that overall I/O writes, it is considered to ask I/O
Be stored in queue, opposite end transmitted data to using multiple threads, process so as to optimize the overall write request of system.
The LVM Remote Replication modules are referred to using in real time to the data consistency definition of remote copy
Synchronous mode, i.e., multiple write requests that upper layer application sends have to wait for main side and the data at standby end are simultaneously written and can continue
Execute；When request arrives, first daily record is submitted to according to request order, write data disks again, when writing data disks, press after writing daily record success
Disk is write successively according to order, after only writing daily record and data disks being write while successfully submitting to, just submits sequential update daily record to according to request
Information, finally notifies upper layer request to complete；
The LVM Remote Replication modules are completed based on NBD modules to the remote transmission of data, right first
Main side and standby end are initialized by nbd_init functions and are connected, then using do_nbd_request functions from request queue
Read corresponding ask, using message structure in handle domains load and be sent to standby end；Call nbd_do_it circular waits simultaneously
Call nbd_read_stat to read for held and write plate-like state；After the completion of read-write requests, then from request queue, removal request is simultaneously
Return successfully to upper strata.
The LVM Remote Replication modules include utilizing tcp_ to the optimization of the parameter of TCP/IP networks
Sack parameters have enabled selection answer-mode, lift the reply of out of order message, only send the message that loses；Using tcp_
Timestamps parameters enable TCP timestamps, and more precise control message is retransmitted；Close tcp_low_latency parameters and selection
Tcp_westwood parameters are optimized to the whole utilization situation of bandwidth by throughput evaluation；Biography for quick long range
The defeated Binary Increase Congestion for enabling tcp_bic parameters, more preferably using the GB step velocitys of transmission network.
Claims (5)
1. the remote copy method based on LVM of a kind of optimization, it is characterised in that：LVM is included based on LVM remote copy frameworks
I/O Analysis modules, three part of LVM I/O Process modules and LVM Remote Replication modules；Described
LVM I/O Analysis modules are optimized to the I/O request analysis of upper layer application, for read request directly accesses local number
According to for write request is then sent to local and remote equipment processing module；During the LVM I/O Process modules are to LVM
Mirror interfaces are optimized, the I/O handling processes of more overwrite request, the method for being updated the data by first log again, will
Data are separately sent to local and remote equipment；The LVM Remote Replication modules are modified to NBD modules,
It is allowed to dock with the inner core of LVM, realizes the long-range transmission of data；Simultaneously as NBD messages are transmitted via TCP/IP
, the LVM Remote Replication modules have been also carried out optimizing to the parameter of TCP/IP networks, adjust TCP/IP nets
Tcp_sack in network, tcp_timestamps, tcp_low_latency, tcp_westwood and tcp_bic parameter, is allowed to
More effectively to suitable for remote copy scene.
2. the remote copy method based on LVM of optimization according to claim 1, it is characterised in that：The LVM I/O
The I/O requests that Analysis modules are responsible for initiating upper layer application are filtered, by changing lvm_make_request_fn () letter
Several to I/O request classify, read request or the request unrelated with duplication directly transfer to LVM self mechanisms to read from disk, write
Request then calls the interface of LVM I/O Process modules to be processed；The LVM I/O Process modules are responsible for building I/
The flow process of O, while dock with Remote Replication modules；The LVM Remote Replication modules are responsible for remote
Data consistency definition and the remote transmission of data that journey is replicated；By I/O requests in main side via LVM Remote
After Replication resume modules, after being sent to remote equipment end and successfully submitting to daily record to write data disks, local day is updated
Will information simultaneously submits to LVM I/O Process modules；When the mirror in LVM I/O Process modules receive local and remote
After the return message of journey, return to application layer and write as work(.
3. the remote copy method based on LVM of optimization according to claim 1 and 2, it is characterised in that：The LVM I/
I/O handling process of the O Process modules to write request, recorded I/O requests in log information first, then that BIOS is multiple
Two parts of system, portion are sent to local data disks, and another is sent in request queue, finally calls Remote
Replication module interfaces, transmit data to opposite end using multiple threads；When receiving local and remote equipment two
After the completion of request, request is returned to upper layer application and processed successfully.
4. the remote copy method based on LVM of optimization according to claim 2, it is characterised in that：The LVM
Remote Replication modules are defined to the data consistency of remote copy and are referred to by the way of real-time synchronization, i.e. upper strata
Multiple write requests that application sends have to wait for main side and the data at standby end are simultaneously written and can continue executing with；When request arrives
When, first daily record is submitted to according to request order, write data disks again, when writing data disks, write disk successively in order after writing daily record success,
After only writing daily record and data disks being write while successfully submitting to, just submit sequential update log information to according to request, finally notify
Layer request is completed；
The LVM Remote Replication modules are completed based on NBD modules to the remote transmission of data, right first
Main side and standby end are initialized by nbd_init functions and are connected, then using do_nbd_request functions from request queue
Read corresponding ask, using message structure in handle domains load and be sent to standby end；Call nbd_do_it circular waits simultaneously
Call nbd_read_stat to read for held and write plate-like state；After the completion of read-write requests, then from request queue, removal request is simultaneously
Return successfully to upper strata.
5. the remote copy method based on LVM of optimization according to claim 1, it is characterised in that：The LVM
Remote Replication modules include having enabled selection using tcp_sack parameters to the optimization of the parameter of TCP/IP networks
Answer-mode, lifts the reply of out of order message, only sends the message that loses；When enabling TCP using tcp_timestamps parameters
Between stab, more precise control message retransmit；Close tcp_low_latency parameters and from tcp_westwood parameters by handling up
Amount assessment is optimized to the whole utilization situation of bandwidth；In order to the transmission of quick long range enables tcp_bic parameters
Binary Increase Congestion, more preferably using the GB step velocitys of transmission network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610913666.5A CN106502828A (en) | 2016-10-20 | 2016-10-20 | A kind of remote copy method based on LVM of optimization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610913666.5A CN106502828A (en) | 2016-10-20 | 2016-10-20 | A kind of remote copy method based on LVM of optimization |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106502828A true CN106502828A (en) | 2017-03-15 |
Family
ID=58318057
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610913666.5A Pending CN106502828A (en) | 2016-10-20 | 2016-10-20 | A kind of remote copy method based on LVM of optimization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106502828A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109002254A (en) * | 2017-06-07 | 2018-12-14 | 华为技术有限公司 | Storage processing method, device and relevant device |
| CN109284202A (en) * | 2017-07-19 | 2019-01-29 | 爱思开海力士有限公司 | Controller and its operating method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070288711A1 (en) * | 2006-06-08 | 2007-12-13 | Inventec Corporation | Snapshot copy management method used for logic volume manager |
| US20080010421A1 (en) * | 2006-07-10 | 2008-01-10 | Inventec Corporation | Snapshot expansion system and method thereof |
| US20090083503A1 (en) * | 2007-09-20 | 2009-03-26 | Inventec Corporation | System of creating logical volume and method thereof |
| US8874746B1 (en) * | 2010-05-24 | 2014-10-28 | Datacore Software Corporation | Collaboration between discrete systems and a shared system to consolidate shared storage-related services |
-
2016
- 2016-10-20 CN CN201610913666.5A patent/CN106502828A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070288711A1 (en) * | 2006-06-08 | 2007-12-13 | Inventec Corporation | Snapshot copy management method used for logic volume manager |
| US20080010421A1 (en) * | 2006-07-10 | 2008-01-10 | Inventec Corporation | Snapshot expansion system and method thereof |
| US20090083503A1 (en) * | 2007-09-20 | 2009-03-26 | Inventec Corporation | System of creating logical volume and method thereof |
| US8874746B1 (en) * | 2010-05-24 | 2014-10-28 | Datacore Software Corporation | Collaboration between discrete systems and a shared system to consolidate shared storage-related services |
Non-Patent Citations (1)
| Title |
|---|
| 葛广贺: "基于LVM2和NBD的远程复制系统设计与性能优化", 《万方数据库》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109002254A (en) * | 2017-06-07 | 2018-12-14 | 华为技术有限公司 | Storage processing method, device and relevant device |
| CN109284202A (en) * | 2017-07-19 | 2019-01-29 | 爱思开海力士有限公司 | Controller and its operating method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11340672B2 (en) | Persistent reservations for virtual disk using multiple targets | |
| US10778767B2 (en) | Persistent memory replication in RDMA-capable networks | |
| CN106657365B (en) | RDMA (remote direct memory Access) -based high-concurrency data transmission method | |
| US11290544B2 (en) | Data transmission methods applied to a proxy server or a backend server, and data transmission system | |
| US11640269B2 (en) | Solid-state drive with initiator mode | |
| CN111277616B (en) | RDMA-based data transmission method and distributed shared memory system | |
| US8233380B2 (en) | RDMA QP simplex switchless connection | |
| WO2014186940A1 (en) | Hard disk and data processing method | |
| US20110004732A1 (en) | DMA in Distributed Shared Memory System | |
| CN109690510A (en) | Multicast device and method for multiple receivers by data distribution into high-performance calculation network and network based on cloud | |
| CN110045912A (en) | Data processing method and device | |
| CN108604163B (en) | Synchronous replication for file access protocol storage | |
| CN103207894A (en) | Multipath real-time video data storage system and cache control method thereof | |
| TW200929950A (en) | Packet forwarding apparatus and method for virtualization switch | |
| JP4208506B2 (en) | High-performance storage device access environment | |
| CN113127139B (en) | Memory allocation method and device based on DPDK of data plane development kit | |
| CN106502828A (en) | A kind of remote copy method based on LVM of optimization | |
| JP2006508459A (en) | High-performance lock management for flash copy in n-way shared storage systems | |
| CN103338156A (en) | Thread pool based named pipe server concurrent communication method | |
| Park et al. | An analysis of replication enhancement for a high availability cluster | |
| US10579310B2 (en) | System and method for reliably persisting storage writes at high speed | |
| CN101807170B (en) | Management method of cache of storage area network | |
| CN110519242A (en) | Data transmission method and device | |
| US11966294B2 (en) | Journal barrier consistency determination | |
| US20050108300A1 (en) | Method for the management of local client cache buffers in a clustered computer environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170315 |
|
| RJ01 | Rejection of invention patent application after publication |