CN105871987A - High available system and method for data writing - Google Patents
High available system and method for data writing Download PDFInfo
- Publication number
- CN105871987A CN105871987A CN201510923305.4A CN201510923305A CN105871987A CN 105871987 A CN105871987 A CN 105871987A CN 201510923305 A CN201510923305 A CN 201510923305A CN 105871987 A CN105871987 A CN 105871987A
- Authority
- CN
- China
- Prior art keywords
- cluster
- data
- write
- standby
- described main
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0668—Management of faults, events, alarms or notifications using network fault recovery by dynamic selection of recovery network elements, e.g. replacement by the most appropriate element after failure
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Abstract
The invention relates to a high available system and method for data writing. The system comprises a main cluster and one or more spare clusters, wherein the main cluster receives a writing operation of a user, and recording the writing operation to a first log file; the main cluster sends a request for synchronizing data to the spare cluster; the spare cluster receives a data synchronization request from the main cluster and writes the data into the spare cluster; and the spare cluster starts a timing deleting mechanism for the wrote data. Through the adoption of the system and method provided by the invention, the memory and the network resource can be saved so as to provide the high availability for data writing, and the complexity of the system can be effectively reduced for the maintenance of a small spare cluster.
Description
[technical field]
The present invention relates to distributed data base field, particularly relate to the high-availability system of a kind of data write
And method.
[background technology]
High availability refers to that system uninterruptedly provides the ability of service, can pass through the mean free error time
(Mean Time Between Failures, MTBF) measures.Described MTBF is computer system
Averagely properly functioning primary fault can the most just occur.The availability of system is the highest, average nothing
Fault time is the longest.
In the environment of current information explodes, every point per second all can produce substantial amounts of data.For ensureing such as
This googol is according to amount, and general employing distributed type assemblies storage scheme is stored in scattered for data in a large number
On independent server.In such cluster, it is general that increase and decrease server, server failure etc. occur
Rate can improve, and the data in cluster now can be caused to need to migrate in a large number.And data migration process
Need to take the resources such as the CPU of server, network interface card, disk and bandwidth.Data Migration once occurs
Situation, the write request that will certainly affect the write of user, even user is rejected completely.But it is existing
Truth condition is, if data can not be written to storage system timely, follow-up for data
Analyze, process operation all will be unable to carry out.This will produce the most negative shadow to the production activity of enterprise
Ring.In this case, the high availability of data write is insufficient for demand.
For improving the write high availability of distributed storage, modal method is that cluster is done full backup,
While i.e. data are written to main frame group, it is synchronized in standby cluster.The most main cluster breaks down, just
Quickly it is switched in standby cluster.The disadvantage of this scheme is to cause memory space and the network bandwidth
Waste.Because under the scene of the Internet, hot spot data can be read several times, and relatively stores relatively
Long data the most in most cases, can't be read into.For very small percentage
Data and waste network and storage resource, the expansion of entreprise cost will certainly be caused.
[summary of the invention]
The technical problem to be solved in the present invention is how to provide a kind of to have obvious cold and hot spy in data
Levy, and when the data of up-to-date generation are had the highest availability requirement, it is possible to achieve not to full dose
Data backup just can ensure the high availability method that hot spot data writes.
For solving above-mentioned technical problem, the present invention provides techniques below scheme
On the one hand, the present invention provides the high-availability system that a kind of data write, and this system includes: main collection
Group and one or more standby cluster, wherein:
Described main cluster receives all write operations of user, and by described operation note to journal file
In；
Described main cluster sends synchrodata to the described request for cluster；
Described standby cluster receives the synchronization request coming from described main cluster, and writes data into described
In standby cluster；And
Described standby cluster starts the timing deleting mechanism to said write data.
On the other hand, the present invention also provides for the high availability method of a kind of data write, is applied to main cluster
And one or more in cluster, the method includes:
Described main cluster receives the write operation of user, and records said write operation to the first daily record literary composition
Part；
Described main cluster sends synchrodata to the described request for cluster；
Described standby cluster receives the data synchronization request coming from described main cluster, and writes described data
To described standby cluster；And
Described standby cluster starts the timing deleting mechanism to write data.
The beneficial effects of the present invention is and be greatly saved storage and Internet resources, be conducive to provide data
The high availability of write.For safeguarding a less standby cluster, can effectively reduce system
Complexity.
[accompanying drawing explanation]
Fig. 1 is the schematic diagram of the high-availability system preferred embodiment of data of the present invention write.
Fig. 2 is the flow chart of the high availability method preferred embodiment of data of the present invention write.
Reference:
| The high-availability system of data write | 1 |
| Main cluster | 2 |
| Standby cluster | 3 |
| Server | 20 |
| Input-output unit | 21 |
| Memory element | 22 |
| Processing unit | 23 |
[detailed description of the invention]
In order to make the purpose of the present invention, technical scheme and advantage clearer, below in conjunction with accompanying drawing
And embodiment, the present invention is further elaborated.Should be appreciated that described herein specifically
Embodiment only in order to explain the present invention, is not intended to limit the present invention.
Refering to shown in Fig. 1, it it is the schematic diagram of the high-availability system preferred embodiment of data of the present invention write.
In the present embodiment, the high-availability system 1 of described data write includes main cluster 2 and
Individual or multiple standby clusters 3.Described main cluster 2 is connected with described standby cluster 3 communication, and described communication connects
Can be cable network communication connection or wireless network communication connection.
In the present embodiment, described main cluster 2 and described standby cluster 3 include one or more clothes respectively
Business device 20 (only illustrating one in figure).Described server 20 can be rack-mount server, blade type
Server, tower server, Cabinet-type server or computer etc. calculate equipment.This server 20 wraps
Include, but be not limited only to, input-output unit 21, memory element 22 and processing unit 23.Described defeated
Entering output unit 21 can be button equipment, such as mouse, keyboard and printer etc., it is also possible to be
Touch-control device, such as LCD (Liquid Crystal Display, liquid crystal) touch screen or OLED
(Organic Light-Emitting Diode, Organic Light Emitting Diode) touch screen etc..These input and output
Unit 21 is for receiving the input operation etc. of user.Described memory element 22 can be described server
20 own internal memories, it is also possible to be SD card (Secure Digital Card, safe digital card), SM
The External memory equipments such as card (Smart Media Card, smart media card), are used for storing described service
The program code installed in device 20 and Various types of data, the such as data etc. of user's write.Described process is single
Unit 23 is for running program code and the computing Various types of data of storage in described memory element 22.
In the present embodiment, the high-availability system 1 of data of the present invention write may be implemented in distributed
Data Migration is carried out, to realize described main cluster 2 and institute between the main cluster of data base 2 and standby cluster 3
State quick backup data between standby cluster 3, improve the high availability of data write.And due to heat
Data have a storage of redundancy, also are able to a certain extent play the effect that calamity is standby.The most main collection
Refer to method embodiments below alternately between group 2 and standby cluster 3, repeat the most one by one.
Refering to shown in Fig. 2, it it is the flow chart of the high availability method preferred embodiment of data of the present invention write.
According to different demands, the execution sequence of the step in the flow chart shown in Fig. 2 can change, some
Step can be omitted.
Step S21, described main cluster 2 receives the write operation of user, and records said write operation extremely
In first journal file.In the present embodiment, described first journal file is for being saved in described main cluster 2
In file.Described first journal file can be stored in distributed file system HDFS, described
First journal file include but not limited to, and the data content of write, described data write described master
The temporal information (or the temporal information being modified) of cluster 2.Described main cluster 2 is defeated by described input
Go out unit 21 and receive the write operation of user.
Step S22, described main cluster 2 sends the request of synchrodata to described standby cluster 3.In this reality
Executing in example, described main cluster 2 starts a process (agent), sends the request of synchrodata to described
Standby cluster 3, and according to described first journal file, in order by the data syn-chronization of write to described standby collection
In group 3.
Step S23, described standby cluster 3 receives the data synchronization request coming from described main cluster 2, and
Write described data to described standby cluster 3.In the present embodiment, described standby cluster 3 starts finger daemon,
Receive the data synchronization request coming from described main cluster 2.
Step S24, described standby cluster 3 starts the timing deleting mechanism to write data.At the present embodiment
In, described timing deleting mechanism is stored in data in described standby cluster 3 for deleting after Preset Time
Mechanism.Described Preset Time configures according to the cold and hot situation of data, such as one month.And when needing
When accessing the data more than a year in a large number, described Preset Time could be arranged to 1 year.At the present embodiment
In, the data of storage before can being deleted relatively early by described timing deleting mechanism, thus safeguard described
Standby cluster 3.
Step S25, when read-write data fault occurs in described main cluster 2, to described main cluster 2 and institute
State standby cluster 3 to switch over.In the present embodiment, when described main cluster 2, read-write data fault occurs
Time, described main cluster 2 and described standby cluster 3 are switched over, described standby cluster 3 is as main cluster 2
Identity data are written and read.
Step S26, described standby cluster 3 receives the write operation that user is current, and records presently written behaviour
Make to the second journal file.In the present embodiment, described second journal file is for being saved in described standby collection
File in group 3.Described second journal file includes as described first journal file, but not only
It is limited to, the described temporal information for cluster 3 of the data content of write, described data write (or by more
The temporal information changed).
Step S27, when the read-write data fault of described main cluster 2 eliminates, described standby cluster 3 synchronizes
Described second journal file is to described main cluster 2, then enters described standby cluster 3 and described main cluster 2 again
Row once switches.In the present embodiment, when the read-write data fault of described main cluster 2 eliminates, institute
State standby cluster 3 and send the request synchronizing described second journal file to described main cluster 2.Described main cluster
2 receive the request synchronizing described second journal file, and again switch described standby cluster 3 and described main collection
Group 2, so can switch back into the identity of described main cluster 2 and described standby cluster 3 again.So,
Described main cluster 2 can receive the write operation of user again, and for the read operation of user, can
To determine from described main cluster 2 according to described first journal file of inquiry and described second journal file
Or described standby cluster 3 reads data.
It addition, during described main cluster 2 is in fault, write needs to the data of described standby cluster 3
Re-write to described main cluster 2.Described standby cluster 3 initiates data synchronization request to described master
Cluster 2, request writes the data to the most described standby cluster 3 to institute during synchronizing described main cluster 2 fault
State main cluster 2.So, if data are not recovered by described main cluster 2 completely, part data i.e. occur
Situation about losing, owing to described standby cluster 3 storing the data in described Preset Time, can be one
Determine to reduce in degree the probability of loss of data.Both ensure that the high availability of write data, also necessarily
The effect that calamity is standby has been reached in degree.
The foregoing is only presently preferred embodiments of the present invention, not in order to limit the present invention, all
Any amendment, equivalent and the improvement etc. made within the spirit and principles in the present invention, all should comprise
Within protection scope of the present invention.
Claims (10)
1. the high-availability system of data write, it is characterised in that this system includes: main cluster with
And one or more for cluster, wherein:
Described main cluster, for receiving the write operation of user, and records said write operation to first
Journal file；And send synchrodata to the described request for cluster；
Described standby cluster, for receiving the data synchronization request coming from described main cluster, and writes institute
State data to described standby cluster；And start the timing deleting mechanism to write data.
2. the high-availability system of data write as claimed in claim 1, it is characterised in that
When read-write data fault occurs in described main cluster, it is current that described standby cluster is additionally operable to receive user
Write operation, and record presently written operation to the second journal file；And/or
When the read-write data fault of described main cluster eliminates, described standby cluster is additionally operable to synchronize described the
Main cluster described in two journal files.
3. the high-availability system of data write as claimed in claim 2, it is characterised in that
Described standby cluster is additionally operable to initiate data synchronization request and synchronizes described master to described main cluster, request
During clustering fault, the data in the most described standby cluster of write are to described main cluster.
4. the high-availability system of data write as claimed in claim 2, it is characterised in that described master
Cluster is additionally operable to when receiving the read requests of user, according to the first journal file of described main cluster and
Second journal file of described standby cluster determines to read number from described main cluster or described standby cluster
According to.
5. the high-availability system of data write as claimed in claim 1, it is characterised in that described fixed
Time deleting mechanism be after Preset Time, delete the data being stored in described standby cluster.
6. a high availability method for data write, is applied to main cluster and one or more standby cluster
In, it is characterised in that the method includes:
Described main cluster receives the write operation of user, and records said write operation to the first daily record literary composition
Part；
Described main cluster sends synchrodata to the described request for cluster；
Described standby cluster receives the data synchronization request coming from described main cluster, and writes described data
To described standby cluster；And
Described standby cluster starts the timing deleting mechanism to write data.
7. the high availability method of data write as claimed in claim 6, it is characterised in that described side
Method also includes:
When read-write data fault occurs in described main cluster, described standby cluster receives the write that user is current
Operation, and record presently written operation to the second journal file；
When the read-write data fault of described main cluster eliminates, the second daily record described in described standby cluster synchronization
Main cluster described in file.
8. the high availability method of data write as claimed in claim 7, it is characterised in that described standby
Cluster is initiated data synchronization request and is write to institute during synchronizing described main clustering fault to main cluster, request
State the data in standby cluster to described main cluster.
9. the high availability method of data write as claimed in claim 8, it is characterised in that described master
When cluster receives the read requests of user, according to the first journal file and the described standby collection of described main cluster
Second journal file of group determines to read data from described main cluster or described standby cluster.
10. the high availability method of data write as claimed in claim 6, it is characterised in that institute
Stating timing deleting mechanism is to delete the data being stored in described standby cluster after Preset Time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510923305.4A CN105871987A (en) | 2015-12-14 | 2015-12-14 | High available system and method for data writing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510923305.4A CN105871987A (en) | 2015-12-14 | 2015-12-14 | High available system and method for data writing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105871987A true CN105871987A (en) | 2016-08-17 |
Family
ID=56624143
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510923305.4A Pending CN105871987A (en) | 2015-12-14 | 2015-12-14 | High available system and method for data writing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105871987A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109101514A (en) * | 2017-06-21 | 2018-12-28 | 北京国双科技有限公司 | Data lead-in method and device |
| CN109408280A (en) * | 2017-08-17 | 2019-03-01 | 北京金山云网络技术有限公司 | Data back up method, apparatus and system |
| CN112000735A (en) * | 2020-08-05 | 2020-11-27 | 中国邮政储蓄银行股份有限公司 | Data processing method, device and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101930468A (en) * | 2010-08-31 | 2010-12-29 | 中兴通讯股份有限公司 | File acquisition method and system |
| CN102088361A (en) * | 2009-12-02 | 2011-06-08 | 成都市华为赛门铁克科技有限公司 | Data backup equipment, method and system |
| CN102867035A (en) * | 2012-08-28 | 2013-01-09 | 浪潮(北京)电子信息产业有限公司 | High-availability method and device of distributed document system cluster |
| CN103593256A (en) * | 2012-08-15 | 2014-02-19 | 阿里巴巴集团控股有限公司 | Method and system for virtual machine snapshot backup on basis of multilayer duplicate deletion |
| CN103645971A (en) * | 2013-12-13 | 2014-03-19 | 江苏名通信息科技有限公司 | File backup and transfer method in LINUX system |
| CN104424283A (en) * | 2013-08-30 | 2015-03-18 | 阿里巴巴集团控股有限公司 | Data migration system and data migration method |
-
2015
- 2015-12-14 CN CN201510923305.4A patent/CN105871987A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102088361A (en) * | 2009-12-02 | 2011-06-08 | 成都市华为赛门铁克科技有限公司 | Data backup equipment, method and system |
| CN101930468A (en) * | 2010-08-31 | 2010-12-29 | 中兴通讯股份有限公司 | File acquisition method and system |
| CN103593256A (en) * | 2012-08-15 | 2014-02-19 | 阿里巴巴集团控股有限公司 | Method and system for virtual machine snapshot backup on basis of multilayer duplicate deletion |
| CN102867035A (en) * | 2012-08-28 | 2013-01-09 | 浪潮(北京)电子信息产业有限公司 | High-availability method and device of distributed document system cluster |
| CN104424283A (en) * | 2013-08-30 | 2015-03-18 | 阿里巴巴集团控股有限公司 | Data migration system and data migration method |
| CN103645971A (en) * | 2013-12-13 | 2014-03-19 | 江苏名通信息科技有限公司 | File backup and transfer method in LINUX system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109101514A (en) * | 2017-06-21 | 2018-12-28 | 北京国双科技有限公司 | Data lead-in method and device |
| CN109408280A (en) * | 2017-08-17 | 2019-03-01 | 北京金山云网络技术有限公司 | Data back up method, apparatus and system |
| CN112000735A (en) * | 2020-08-05 | 2020-11-27 | 中国邮政储蓄银行股份有限公司 | Data processing method, device and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10956601B2 (en) | Fully managed account level blob data encryption in a distributed storage environment | |
| US9916201B2 (en) | Write performance in fault-tolerant clustered storage systems | |
| US8868858B2 (en) | Method and apparatus of continuous data backup and access using virtual machines | |
| US10659225B2 (en) | Encrypting existing live unencrypted data using age-based garbage collection | |
| US20190007206A1 (en) | Encrypting object index in a distributed storage environment | |
| CN102024044B (en) | Distributed file system | |
| US9477565B2 (en) | Data access with tolerance of disk fault | |
| CN103345470B (en) | A kind of database disaster recovery method, system and server | |
| US10719407B1 (en) | Backing up availability group databases configured on multi-node virtual servers | |
| US20150213100A1 (en) | Data synchronization method and system | |
| CN103516736A (en) | Data recovery method of distributed cache system and a data recovery device of distributed cache system | |
| US9753792B2 (en) | Method and system for byzantine fault tolerant data replication | |
| CN103268318A (en) | Distributed key value database system with strong consistency and read-write method thereof | |
| US8566636B2 (en) | Application recovery in a file system | |
| KR20100070968A (en) | Cluster data management system and method for data recovery using parallel processing in cluster data management system | |
| US20120278429A1 (en) | Cluster system, synchronization controlling method, server, and synchronization controlling program | |
| CN103037004A (en) | Implement method and device of cloud storage system operation | |
| CN101901250A (en) | Memory database and data processing method thereof | |
| US8527454B2 (en) | Data replication using a shared resource | |
| CN102110154A (en) | File redundancy storage method in cluster file system | |
| CN103516549A (en) | File system metadata log mechanism based on shared object storage | |
| US8683258B2 (en) | Fast I/O failure detection and cluster wide failover | |
| CN105871987A (en) | High available system and method for data writing | |
| CN108259569A (en) | It is a kind of based on IPSAN share storage without acting on behalf of continuous data protection method | |
| CN116204137B (en) | Distributed storage system, control method, device and equipment based on DPU |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20160817 |