CN109542674A - Snapshot creation method, device, equipment and the medium of distributed system - Google Patents
Snapshot creation method, device, equipment and the medium of distributed system Download PDFInfo
- Publication number
- CN109542674A CN109542674A CN201710861846.8A CN201710861846A CN109542674A CN 109542674 A CN109542674 A CN 109542674A CN 201710861846 A CN201710861846 A CN 201710861846A CN 109542674 A CN109542674 A CN 109542674A
- Authority
- CN
- China
- Prior art keywords
- write request
- request
- snapshot
- distributed system
- receiving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
Abstract
The embodiment of the present application discloses snapshot creation method, device, equipment and the medium of a kind of distributed system, to while generating distributed system snapshot, response user is directed to the write request of distributed system.The snapshot creation method of the distributed system, comprising: receive the write request for being directed to the distributed system；If the write request receives after receiving the snapshot for the distributed system and generating request, the original metadata for recording object storage location in the distributed system is backed up, backup metadata is obtained；The write request is executed for the original metadata, the snapshot is executed for the backup metadata and generates request.
Description
Technical field
This application involves snapping technique field more particularly to a kind of snapshot creation methods of distributed system, device, equipment
And computer readable storage medium.
Background technique
With the development of cloud computing technology, more and more users complete data calculating and data using cloud computing service
Storage, the reliability of data are particularly important to user.User is often desirable to that in data calculation process, snapshot can be generated,
To guarantee not losing historical data or can audit to the state of historical data.
Traditional distributed system snapshot generates scheme, in order to guarantee the consistency of snapshot, it is desirable that receives snapshot generation and asks
After asking all write requests (user changes the request of data mode, for example, write-in data, modification data, delete data etc.,
Hereinafter referred " write request ") to the change of data mode not in snapshot, and receive all before snapshot generates request
Write request is to the change of data mode in snapshot.Therefore, it usually needs to block use after receiving snapshot and generating request
The write request at family is completed for example, no longer receiving the write request of user or no longer responding the write request of user until snapshot generates
And then restore the reception or response of user's write request.
However, being blocked during being completed after receiving snapshot and generating request to snapshot generation to the write request of user
The normal use of user certainly will be will affect.
Summary of the invention
The embodiment of the present application provides the snapshot creation method of distributed system a kind of, device, equipment and computer-readable
Storage medium, to while generating distributed system snapshot, response user is directed to the write request of distributed system.
According to the one side of the embodiment of the present application, a kind of snapshot creation method of distributed system is provided, comprising:
Receive the write request for being directed to distributed system；
If write request receives after receiving the snapshot for distributed system and generating request, to the distributed system of record
The original metadata of object storage location is backed up in system, obtains backup metadata；
The write request received after receiving snapshot generation request is executed for original metadata, for backup metadata
It executes snapshot and generates request.
According to the another aspect of the embodiment of the present application, a kind of snapshot generating means of distributed system are provided, comprising:
Receiving module, for receiving the write request for being directed to distributed system；
Metadata backup module, for determining write request after receiving the snapshot for distributed system and generating request
When receiving, the original metadata of object storage location in record distributed system is backed up, backup metadata is obtained；
Processing module, for executing the write request received after receiving snapshot generation request for original metadata,
Snapshot, which is executed, for backup metadata generates request.
According to the another aspect of the embodiment of the present application, a kind of snapshot generating device of distributed system is provided, comprising: storage
Device and processor；The memory is for storing executable program code；The processor be used for read stored in memory can
Program code is executed to execute the snapshot creation method of above-mentioned distributed system.
According to the embodiment of the present application in another aspect, provide a kind of computer readable storage medium, it is stored thereon with calculating
Machine program instruction realizes the snapshot creation method of above-mentioned distributed system when computer program instructions are executed by processor.
According to the snapshot creation method of the distributed system in the embodiment of the present application, device, equipment and computer-readable deposit
Storage media, after receiving the write request for distributed system, however, it is determined that write request is being received for distributed system
Snapshot generates what request received later, then carries out to the original metadata of object storage location in record distributed system standby
Part, backup metadata is obtained, and the write request received after receiving snapshot generation request, needle are executed for original metadata
Snapshot is executed to backup metadata and generates request.The snapshot of the embodiment of the present application generates scheme, by record distributed system
The original metadata of middle object storage location is backed up, and backup metadata is obtained, and makes it possible to execute for original metadata
The write request received after request is generated receiving snapshot, while executing snapshot for backup metadata and generating request, namely
It realizes while generating distributed system snapshot, response user is directed to the write request of distributed system.
Detailed description of the invention
In order to more clearly explain the technical solutions in the embodiments of the present application, make required in being described below to embodiment
Attached drawing is briefly introduced, it should be apparent that, the drawings in the following description are only some examples of the present application, for this
For the those of ordinary skill in field, without creative efforts, it can also be obtained according to these attached drawings other
Attached drawing.
Fig. 1 shows the structural schematic diagram of cloud computing service system architecture；
Fig. 2 shows the signals according to the application scenarios of the snapshot creation method of the distributed system of the embodiment of the present application
Figure；
Fig. 3 shows the schematic illustration for adding label in write request according to one of the embodiment of the present application；
Fig. 4 shows another schematic illustration that label is added in write request according to the embodiment of the present application；
Fig. 5 shows the flow diagram of the snapshot creation method of the distributed system according to the embodiment of the present application；
Fig. 6 shows the structural schematic diagram of the snapshot generating means according to the distributed system of the embodiment of the present application；
Fig. 7 shows the meter that can be realized snapshot creation method and device according to the distributed system of the embodiment of the present application
Calculate the structure chart of the exemplary hardware architecture of equipment.
Specific embodiment
The feature and exemplary embodiment of the various aspects of the application is described more fully below, in order to make the mesh of the application
, technical solution and advantage be more clearly understood, with reference to the accompanying drawings and embodiments, the application is further retouched in detail
It states.It should be understood that specific embodiment described herein is only configured to explain the application, it is not configured as limiting the application.
To those skilled in the art, the application can be real in the case where not needing some details in these details
It applies.Below the description of embodiment is used for the purpose of better understanding the application to provide by showing the example of the application.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence " including ... ", it is not excluded that including
There is also other identical elements in the process, method, article or equipment of the element.
It should be noted that since read request of the user to data in distributed system (reads number from distributed system
According to state, referred to as " read request ") change distributed system in data state, therefore, to data in distributed system
Read request can with generate snapshot carry out simultaneously, namely to generate distributed system snapshot when, do not influence to distribution
The normal execution of data read request in system.The embodiment of the present application is mainly illustrated by taking write request as an example, still, this field skill
Art personnel it should be understood that in the embodiment of the present application related to write request part, can be extended to and read request and write
Request.
Traditional snapshot generates scheme, in order to guarantee the consistency of snapshot, it is desirable that receives the institute after snapshot generation request
There is write request to the change of data mode not in snapshot, while receiving snapshot and generating all write request logarithms before request
According to the change of state in snapshot.Therefore, it is generated after receiving snapshot and generating request to snapshot during completing, it can be to user
Write request blocked, and this certainly will will affect the normal use of user, for some applications for needing to be continuously written into data
Scene, traditional snapshot generate scheme and cannot be applicable in.
Inventor has found during realizing the application, after user submits a write request, if system not to
User returns to the completed calculated result of the write request, and it is complete to the change of data mode that user is not generally regarded as this write request
At, namely before system does not return to the completed calculated result of the request to user, user is not aware that this write request
To the progress of data mode change.
Based on this, inventors have found that in a distributed system, user asks for receiving snapshot generation in distributed system
It is submitting before asking and receive snapshot in distributed system and generate not yet to return after request and writing for calculated result is completed asks
It asks, since user is not aware that the progress that the write request changes data mode, user is not to be especially concerned about that this is write to ask
Ask the change to data mode whether in snapshot.
In view of this, the snapshot that the embodiment of the present application provides a kind of distributed system generates scheme, receive for distribution
The write request of formula system；If write request receives after receiving the snapshot for distributed system and generating request, to record
The original metadata of object storage location is backed up in distributed system, obtains backup metadata；It is held for original metadata
The write request that row receives after receiving snapshot generation request, executes snapshot for backup metadata and generates request.
The snapshot of distributed system provided by the embodiments of the present application generates scheme, by object in record distributed system
The original metadata of storage location is backed up, and backup metadata is obtained, and makes it possible to receiving for original metadata execution
Snapshot generates the write request received after request, while executing snapshot for backup metadata and generating request, namely realizing
While generating distributed system snapshot, response user is directed to the write request of distributed system.
In one example, the write request for being directed to distributed system is received, which can be user and be sent directly to
Distributed system, it is also possible to user by other equipment and is sent to distributed system, the application does not limit this.Its
In, distributed system can include but is not limited to: distributed file system, distributed memory system, distributed data base system
Deng.
It in one embodiment, in order to facilitate the determining write request received is sent out before receiving snapshot and generating request
The write request sent, or the write request sent after request is generated receiving snapshot, if user can know that distributed system connects
At the time of receiving snapshot generation request, then user can receive when sending write request to distributed system in distributed system
Preset mark is added in the write request sent after generating the request moment to snapshot, receives snapshot to facilitate distributed system to distinguish
Generate the write request of request front and back.
In one embodiment, special symbol or character can be used as preset mark, also can be used to write and ask
Middle finger is asked to determine the numerical value of data bit as preset mark.
For example, when using special symbol or character as preset mark, preset mark is " * * * ", is added with
The write request of " * * * " is that distributed system receives the write request sent after snapshot generation request, and does not include in write request
" * * * " is the write request that distributed system receives that snapshot generates transmission before request.
For another example, when write request middle finger being used to determine the numerical value of data bit as preset mark, write request middle finger determines data bit
Numerical value when being the first numerical value, indicate the write request be distributed system receive snapshot generate send after request write and ask
It asks, write request middle finger determines the numerical value of data bit when being second value, and it is raw to indicate that the write request is that distributed system receives snapshot
At the write request sent before request.Wherein, the first numerical value is only used for distinguishing two different numerical value from second value, for example,
First numerical value is 1, second value 0.
It certainly, can also be by user if user can not know that distributed system receives snapshot generation request
The other equipment or other modules in distributed system for forwarding write request between distributed system, connect in distributed system
It receives in the write request sent after snapshot generation request and adds preset mark, the application does not limit this.
In one embodiment, it is generated in the write request sent after request due to receiving snapshot in distributed system
Added with preset mark, therefore, after receiving the write request for distributed system, whether can be wrapped according in write request
Containing preset mark, judges whether write request receives after snapshot generates request in distributed system and receive.
In one example, however, it is determined that include preset mark in write request, it is determined that write request is received in distributed system
It is received after generating request to snapshot；If it is determined that not including preset mark in write request, it is determined that write request is in distribution system
System is received before snapshot generates request and is received.
For example, the example that above-mentioned preset mark is " * * * " is still continued to use, after receiving write request, however, it is determined that write and ask
Include in asking " * * * ", it is determined that write request is received after snapshot generates request in distributed system and received；It is asked if it is determined that writing
Do not include in asking " * * * ", it is determined that write request is received before snapshot generates request in distributed system and received.
It can be carried out simultaneously to guarantee to generate snapshot with write request, and guarantee to receive all after snapshot generation is requested
Write request not in snapshot, receives snapshot and generates all write requests before request to data mode to the change of data mode
Change in snapshot.In one embodiment, for the write request received after receiving snapshot generation request, then
The original metadata of object storage location in record distributed system is backed up, backup metadata is obtained, for original member
Data execute the write request received after receiving snapshot generation request；For being received before receiving snapshot and generating request
Write request, execute the write request that receives before receiving snapshot and generating request for backup metadata.Wherein, distributed system
Object in system, can include but is not limited to: the file in data, distributed system in distributed system.
In one embodiment, the write request received after receiving snapshot generation request, comprising: receiving snapshot
The write request submitted for the first time after request is generated, and/or is submitted before receiving snapshot and generating request and executes failure and mentioned again
The write request of friendship.
It in one example, is to generate request in reception snapshot receiving snapshot to generate the write request received after request
When the write request submitted for the first time later, change of the write request to data mode in database, there can be no in snapshot, because
This executes the write request for original metadata when executing the write request.
It in one example, is to generate request in reception snapshot receiving snapshot to generate the write request received after request
Before submit and execute unsuccessfully resubmit write request when, although the write request receive snapshot generate request before once
It submitted, but was resubmited after receiving snapshot and generating request since it executes failure, then this is write and ask
Seek the write request for being considered as and submitting after receiving snapshot generation request.Therefore, when executing the write request, for original first number
According to the execution write request.Namely the write request does not appear in snapshot the change of data mode.
In one embodiment, executed for original metadata receive snapshot generate receive after request write and ask
When asking, the write request for receiving and receiving before snapshot generation request can be executed for backup metadata, and first for backup
Data execute snapshot and generate request.
In one embodiment, if in write request including the Data Identification of data to be changed, determining that write request is
It receives snapshot and generates the write request received after request, to the original metadata of object storage location in record distributed system
It is backed up, when obtaining backup metadata, can first determine the data block of storing data mark corresponding data in distributed system,
And the original metadata of object storage location in record data block is backed up, backup metadata is obtained, to reduce calculation amount.
Wherein, database is by multiple data chunks at data block is made of multiple storage units, and data are stored in a storage unit.
In one embodiment, fast to read when reading the data in distributed system after generating snapshot
Data according in can then be routed based on backup metadata, and to read real time data, then it can be based on original first number
According to being routed.
Currently, in order to provide a user the unrestricted data service of memory capacity, often being adopted in cloud computing service
With the calculating system architecture isolated with storage.
As shown in Figure 1, the program (for example, MySQL calculate node) or process of user operates in cloud computing service
In host (host 1, host 2 and host 3 shown in fig. 1).Cloud virtual disk interchanger (Cloud Virtual on host
Disk Switch) disk (namely cloud disk 10) is simulated in host, it is accessed for user program, while making user's
Data can be stored in cloud disk 10 by communication network.The memory space of cloud disk 10 is unrestricted, the data to user into
When row physical store, the data in cloud disk 10 are stored in distributed memory system.
For example, cloud disk 10 can be divided into multiple data blocks (data block 1 shown in Fig. 1, data block 2, data block
3 ... ..., data block N, wherein N is natural number), each data block can be divided into multiple storage units again, finally with storage
The data of user are stored in the distributed storage server (storage shown in Fig. 1 independently of host deployments for unit by unit
Server 1, storage server 2 and storage server 3) in.The provider of cloud computing service can be right by cloud disk console 11
Data in the host and cloud disk 10 of 10 carry of cloud disk are managed, for example, generating the snapshot of data in cloud disk 10.Above system
There can be one or more hosts in framework, there can also be one or more storage servers.
In one embodiment, host can be movable equipment.For example, it may be mobile phone, tablet computer etc..Host
It can also be desk device, such as: all-in-one machine etc..
In one embodiment, cloud virtual disk interchanger receives the read request and/or write request that user sends, and root
Corresponding storage server is routed requests to according to the offset (offset) in read request and/or write request.
In one embodiment, cloud disk 10 can be simultaneously by multiple main frames carry, but 10 permission quilts of cloud disk
For one host with read-write mode carry, other hosts can only be with read-only mode carry.
In the system architecture of cloud computing service, as shown in Fig. 2, the provider (user 20) of cloud computing service can be in cloud
The snapshot that disk console 11 initiates to generate data in cloud disk 10 snapshot generates request, and snapshot, which generates, carries cloud disk mark in request
Know.Wherein, cloud disk mark all has unique cloud disk mark for distinguishing different cloud disks, each cloud disk.For example, cloud disk
Mark can be the sequence number of cloud disk.
After cloud disk console 11 receives the snapshot generation request comprising cloud disk mark, is generated in request and wrapped according to snapshot
The cloud disk mark contained, determines the cloud disk for needing to generate snapshot, for example, determining that the cloud disk for needing to generate snapshot is to show in Fig. 1
Cloud disk 10.Determine need to generate snapshot cloud disk be cloud disk 10 after, due to read-write mode carry in cloud disk 10
Host may the change to the data mode in cloud disk 10 in generating snapshot processes, therefore, cloud disk console 11, which determines, to be hung
The carry mode of the host and each host that are loaded in cloud disk 10, and asked to snapshot generation is sent with the host of read-write mode carry
It asks, so as to know to generate snapshot to the data in cloud disk 10 with the host of read-write mode carry.
For example, host 1, host 2 and the equal carry of host 3 are in cloud disk 10 in Fig. 1, if host 1 is hung with read-only mode
It carries, host 2 and host 3 are determining that the cloud disk for needing to generate snapshot is cloud disk in cloud disk console 11 with read-only mode carry
10, and determine host 1 with read-only mode carry in cloud disk 10 after, to host 1 send snapshot generate request.
After host 1 receives snapshot generation request, at the time of record receives snapshot generation request, if host 1 is connecing
After receiving at the time of snapshot generation is requested, the write request of the transmission of user 21 is received, then host 1 can add in the write request
Adding for identifying the write request is the label for receiving snapshot and generating the write request received after request, and will be added with label
Write request be forwarded to storage server.For example, " * * * " is added in write request, is to receive for identifying write request
Snapshot generates the write request received after request.
In one embodiment, host 1 according to receive snapshot generate request at the time of with receive write request when
Relationship between quarter judges whether write request is the write request received after snapshot generation request.
In one example, if host 1 determine receive snapshot generate request at the time of earlier than receive write request when
It carves, then host 1 determines that the write request is the write request received after snapshot generation request；If the determination of host 1 receives snapshot
It is later than at the time of receive write request at the time of generating request, then host 1 determines that the write request is to receive snapshot to generate request
Write request before.
For example, as shown in figure 3, having moment A, moment B, moment C, moment D, moment E and moment on moment axis
F totally six moment, and moment F is later than moment A, if host 1 receives write request 1 in moment A, receives write request in moment B
2, write request 3 is received in moment C, snapshot is received in moment D and generates request, write request 4 is received in moment E, in moment F
Receive write request 5, then host 1 determine the time of reception of write request 4 and write request 5 be later than receive snapshot generate request when
It carves, host 1 is to receive after snapshot generates request to receive by addition is used to identify write request in write request 4 and write request 5
The label of the write request arrived.
As shown in figure 4, if executing failure in the write request 2 that moment B is submitted, G at the time of after moment F, 21 weight of user
Write request 2 newly is had submitted, then host 1 determines that the time of reception for the write request 2 that moment G is submitted is later than and receives snapshot generation request
At the moment, it is to receive snapshot to generate being followed by for request that host 1, which is added in the write request 2 that moment G is submitted for identifying write request,
The label of the write request received.
After storage server receives the markd write request of addition, determine that the write request generates request to receive snapshot
The write request received later, then in duplicated record storage server data storage location original metadata, generate backup member
Data, and the markd write request of addition received is executed for original metadata.
In one embodiment, after storage server receives the markd write request of addition, if being taken in write request
With Data Identification, storage server can first determine the data block for storing above-mentioned Data Identification corresponding data, and duplicated record
The original metadata of data storage location in above-mentioned data block generates backup metadata, and executes reception for original metadata
The markd write request of the addition arrived.
In one embodiment, storage server executes for original metadata and receives being followed by for snapshot generation request
When the write request received, the write request for receiving and receiving before snapshot generation request can be executed for backup metadata, and
Snapshot is executed for backup metadata and generates request, is realized while generating snapshot, is responded the write request of user.
It should be noted that snapshot provided by the embodiments of the present application generates scheme, it is applicable not only to distributed system, it is same
The snapshot that sample is suitable for other data-storage systems generates.For example, the application reality also can be used in standalone storage system
The snapshot for applying example generates scheme.
Below with reference to specific system process flow to the implementation procedure of the snapshot creation method of above-mentioned distributed system into
Row explanation, it should be noted, however, that the specific embodiment merely to the application is better described, is not constituted to this Shen
Improper restriction please.
For overall flow, as shown in figure 5, the snapshot creation method 500 of distributed system, may include following step
It is rapid:
Step S501 receives the write request for being directed to distributed system.
Step S502, if write request receives after receiving the snapshot for distributed system and generating request, to note
The original metadata of object storage location is backed up in record distributed system, obtains backup metadata.
Step S503 executes the write request received after receiving snapshot generation request for original metadata, for
Backup metadata executes snapshot and generates request.
According in the embodiment of the present application, carried out by the original metadata to object storage location in record distributed system
Backup, obtains backup metadata, makes it possible to receive after receiving snapshot and generating request for original metadata execution
Write request, while executing snapshot for backup metadata and generating request, namely realizing and generating the same of distributed system snapshot
When, response user is directed to the write request of distributed system.
When realizing, the snapshot creation method of the distributed system in above-described embodiment can pass through distributed system
Snapshot generating means realize.As shown in fig. 6, the snapshot generating means 600 of distributed system, may include:
Receiving module 601, for receiving the write request for being directed to distributed system.
Metadata backup module 602, for determining snapshot generation request of the write request in reception for distributed system
When receiving later, the original metadata of object storage location in record distributed system is backed up, obtains backing up first number
According to.
Processing module 603, for executed for original metadata received after receiving snapshot and generating request write and ask
It asks, executes snapshot for backup metadata and generate request.
In one embodiment, metadata backup module 602, is also used to: determining that write request is being received for distribution
The snapshot of system receives before generating request；Processing module 603, is also used to: executing for backup metadata and is receiving snapshot
Generate the write request received before request.
In one embodiment, receiving module 601 are specifically used for: receiving the write request for being directed to distributed system；According to
Whether include preset mark in write request, judges whether write request receives after receiving snapshot and generating request.
In one embodiment, receiving module 601 are specifically used for: if it is determined that including preset mark in write request, then really
Determine write request to receive after receiving snapshot and generating request；If it is determined that not including preset mark in write request, it is determined that write and ask
It asks and is received before receiving snapshot and generating request.
In one embodiment, distributed system includes any of the following or a variety of: distributed file system is divided
Cloth storage system, distributed data base system.
In one embodiment, receiving module 601 are specifically used for: receive calculating equipment transmission is directed to distributed system
The write request of system is calculated equipment and is connected with distributed system with read-write mode.
In one embodiment, calculate equipment be according to write request whether receive snapshot generate request after receive
It arrives, judges whether to add preset mark in write request.
In one embodiment, calculating equipment is at the time of generating request according to reception snapshot and when receiving write request
Relationship between quarter, judges whether write request receives after receiving snapshot and generating request.
It in one embodiment, include the Data Identification of data to be changed in write request；Metadata backup module 602,
It is specifically used for: determines the data block of storing data mark corresponding data in distributed system, and object in record data block is deposited
The original metadata that storage space is set is backed up, and backup metadata is obtained.
In one embodiment, the write request received after receiving snapshot generation request, comprising: receiving snapshot
The write request submitted for the first time after request is generated, and/or is submitted before receiving snapshot and generating request and executes failure and mentioned again
The write request of friendship.
Fig. 7 shows the meter that can be realized snapshot creation method and device according to the distributed system of the embodiment of the present application
Calculate the structure chart of the exemplary hardware architecture of equipment.As shown in fig. 7, calculating equipment 700 includes input equipment 701, input interface
702, central processing unit 703, memory 704, output interface 705 and output equipment 706.Wherein, input interface 702, center
Processor 703, memory 704 and output interface 705 are connected with each other by bus 710, input equipment 701 and output equipment
706 are connect by input interface 702 and output interface 705 with bus 710 respectively, and then the other assemblies with calculating equipment 700
Connection.
Specifically, input equipment 701 is received from external input information, and will input information by input interface 702
It is transmitted to central processing unit 703；Central processing unit 703 is based on the computer executable instructions stored in memory 704 to input
Information is handled to generate output information, and output information is temporarily or permanently stored in memory 704, is then passed through
Output information is transmitted to output equipment 706 by output interface 705；Output information is output to and calculates equipment 700 by output equipment 706
Outside for users to use.
That is, the equipment shown in Fig. 7 that calculates also may be implemented as the snapshot generating device of distributed system, this point
The snapshot generating device of cloth system may include: the memory for being stored with computer executable instructions；And processor, at this
Reason device may be implemented to combine the snapshot creation method of the distributed system of Fig. 1-Fig. 6 description when executing computer executable instructions
And device.
In the above-described embodiments, can come wholly or partly by software, hardware, firmware or any combination thereof real
It is existing.It when implemented in software, can be entirely or partly with the shape of computer program product or computer readable storage medium
Formula is realized.The computer program product or computer readable storage medium include one or more computer instructions.It is calculating
When being loaded on machine and executing the computer program instructions, entirely or partly generate according to process described in the embodiment of the present application
Or function.The computer can be general purpose computer, special purpose computer, computer network or other programmable devices.Institute
Stating computer instruction may be stored in a computer readable storage medium, or from a computer readable storage medium to another
A computer readable storage medium transmission, for example, the computer instruction can be from web-site, computer, a server
Or data center passes through wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (for example, infrared, wireless, micro-
Wave etc.) mode transmitted to another web-site, computer, server or data center.The computer-readable storage
Medium can be any usable medium that computer can access or include the integrated service of one or more usable mediums
The data storage devices such as device, data center.The usable medium can be magnetic medium, (for example, floppy disk, hard disk, tape), light
Medium (for example, DVD) or semiconductor medium (for example, solid state hard disk Solid State Disk (SSD)) etc..
In addition, in conjunction with the snapshot creation method of the distributed system in above-described embodiment, the embodiment of the present application can provide one
Computer readable storage medium is planted to realize.Computer program instructions are stored on the computer readable storage medium；The calculating
Machine program instruction realizes the snapshot creation method of any one distributed system in above-described embodiment when being executed by processor.
It should be clear that the application is not limited to specific configuration described above and shown in figure and processing.
For brevity, it is omitted here the detailed description to known method.In the above-described embodiments, several tools have been described and illustrated
The step of body, is as example.But the present processes process is not limited to described and illustrated specific steps, this field
Technical staff can be variously modified, modification and addition after understanding spirit herein, or suitable between changing the step
Sequence.
It should also be noted that, the exemplary embodiment referred in the application, is retouched based on a series of step or device
State certain methods or system.But the application is not limited to the sequence of above-mentioned steps, that is to say, that can be according in embodiment
The sequence referred to executes step, may also be distinct from that the sequence in embodiment or several steps are performed simultaneously.
The above, the only specific embodiment of the application, it is apparent to those skilled in the art that,
For convenience of description and succinctly, the system, module of foregoing description and the specific work process of unit can refer to preceding method
Corresponding process in embodiment, details are not described herein.It should be understood that the protection scope of the application is not limited thereto, it is any to be familiar with
Those skilled in the art within the technical scope of the present application, can readily occur in various equivalent modifications or substitutions,
These modifications or substitutions should all cover within the scope of protection of this application.
Claims (22)
1. a kind of snapshot creation method of distributed system, which is characterized in that the described method includes:
Receive the write request for being directed to the distributed system；
If the write request receives after receiving the snapshot for the distributed system and generating request:
The original metadata for recording object storage location in the distributed system is backed up, backup metadata is obtained；
The write request is executed for the original metadata, the snapshot is executed for the backup metadata and generates request.
2. the method according to claim 1, wherein the method also includes:
If the write request receives before receiving the snapshot for the distributed system and generating request, for described standby
Part metadata executes the write request.
3. the method according to claim 1, wherein it is described receive be directed to the distributed system write request,
Include:
Receive the write request for being directed to the distributed system；
According to whether including preset mark in the write request, judge whether the write request is receiving the snapshot generation request
It receives later.
4. according to the method described in claim 3, it is characterized in that, whether described include pre- bidding according in the write request
Note, judges whether the write request receives after receiving the snapshot and generating request, comprising:
If it is determined that including preset mark in the write request, it is determined that the write request is after receiving the snapshot and generating request
It receives；
If it is determined that not including preset mark in the write request, it is determined that the write request is receiving the snapshot generation request
Before receive.
5. the method according to claim 1, wherein the distributed system, includes any of the following or more
Kind: distributed file system, distributed memory system, distributed data base system.
6. according to the method described in claim 3, it is characterized in that, it is described receive be directed to the distributed system write request,
Include:
It receives and calculates the write request for the distributed system that equipment is sent, the calculating equipment and the distributed system
It is connected with read-write mode.
7. according to the method described in claim 6, it is characterized in that, whether the calculating equipment is being connect according to the write request
It receives after the snapshot generates request and receives, judge whether to add the preset mark in the write request.
8. the method according to the description of claim 7 is characterized in that the calculating equipment is asked according to the reception snapshot generation
Relationship between at the time of asking and at the time of the reception write request, judges whether the write request is receiving the snapshot generation
It is received after request.
9. method according to claim 1 to 8, which is characterized in that
It include the Data Identification of data to be changed in the write request；
The described pair of original metadata for recording object storage location in the distributed system backs up, and obtains backing up first number
According to, comprising:
Determine the data block that the Data Identification corresponding data is stored in the distributed system, and to recording in the data block
The original metadata of object storage location is backed up, and backup metadata is obtained.
10. method according to claim 1 to 8, which is characterized in that described to be asked in the reception snapshot generation
The write request received after asking, comprising:
The write request submitted for the first time after request is generated receiving the snapshot, and/or generates request receiving the snapshot
The write request that preceding submission and execution are unsuccessfully resubmited.
11. a kind of snapshot generating means of distributed system, which is characterized in that described device includes:
Receiving module, for receiving the write request for being directed to the distributed system；
Metadata backup module, for determining snapshot generation request of the write request in reception for the distributed system
When receiving later, the original metadata for recording object storage location in the distributed system is backed up, is backed up
Metadata；
Processing module, for executing the write request for the original metadata, for described in backup metadata execution
Snapshot generates request.
12. device according to claim 11, which is characterized in that
The metadata backup module, is also used to: determining that the write request is receiving the snapshot life for the distributed system
It is received before at request；
The processing module, is also used to: executing the write request for the backup metadata.
13. device according to claim 11, which is characterized in that the receiving module is specifically used for:
Receive the write request for being directed to the distributed system；
According to whether including preset mark in the write request, judge whether the write request is receiving the snapshot generation request
It receives later.
14. device according to claim 13, which is characterized in that the receiving module is specifically used for:
If it is determined that including preset mark in the write request, it is determined that the write request is after receiving the snapshot and generating request
It receives；
If it is determined that not including preset mark in the write request, it is determined that the write request is receiving the snapshot generation request
Before receive.
15. device according to claim 11, which is characterized in that the distributed system, include any of the following or
It is a variety of: distributed file system, distributed memory system, distributed data base system.
16. device according to claim 13, which is characterized in that the receiving module is specifically used for:
It receives and calculates the write request for the distributed system that equipment is sent, the calculating equipment and the distributed system
It is connected with read-write mode.
17. device according to claim 16, which is characterized in that the calculating equipment be according to the write request whether
It receives after the snapshot generates request and receives, judge whether to add the preset mark in the write request.
18. device according to claim 17, which is characterized in that the calculating equipment is generated according to the reception snapshot
Relationship between at the time of request and at the time of the reception write request, judges whether the write request is receiving the snapshot life
It is received later at request.
19. device described in any one of 1-18 according to claim 1, which is characterized in that
It include the Data Identification of data to be changed in the write request；
The metadata backup module, is specifically used for:
Determine the data block that the Data Identification corresponding data is stored in the distributed system, and to recording in the data block
The original metadata of object storage location is backed up, and backup metadata is obtained.
20. device described in any one of 1-18 according to claim 1, which is characterized in that described to receive the snapshot generation
The write request received after request, comprising:
The write request submitted for the first time after request is generated receiving the snapshot, and/or generates request receiving the snapshot
The write request that preceding submission and execution are unsuccessfully resubmited.
21. a kind of snapshot generating device of distributed system, which is characterized in that including memory and processor；The memory
For storing executable program code；The processor be used to read the executable program code that is stored in the memory with
Perform claim requires method described in any one of 1-10.
22. a kind of computer readable storage medium, is stored thereon with computer program instructions, which is characterized in that when the calculating
Such as method of any of claims 1-10 is realized when machine program instruction is executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710861846.8A CN109542674A (en) | 2017-09-21 | 2017-09-21 | Snapshot creation method, device, equipment and the medium of distributed system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710861846.8A CN109542674A (en) | 2017-09-21 | 2017-09-21 | Snapshot creation method, device, equipment and the medium of distributed system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109542674A true CN109542674A (en) | 2019-03-29 |
Family
ID=65823428
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710861846.8A Pending CN109542674A (en) | 2017-09-21 | 2017-09-21 | Snapshot creation method, device, equipment and the medium of distributed system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109542674A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110765071A (en) * | 2019-11-08 | 2020-02-07 | 无锡华云数据技术服务有限公司 | Method and related device for carrying out snapshot on distributed storage |
| CN112965945A (en) * | 2019-12-13 | 2021-06-15 | 阿里巴巴集团控股有限公司 | Data storage method and device, electronic equipment and computer readable medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1704903A (en) * | 2004-06-03 | 2005-12-07 | 华为技术有限公司 | Snapshot backup method |
| CN102971698A (en) * | 2012-06-29 | 2013-03-13 | 华为技术有限公司 | Snapshot data-processing method and system, storage system and snapshot agency |
| US8818951B1 (en) * | 2011-12-29 | 2014-08-26 | Emc Corporation | Distributed file system having separate data and metadata and providing a consistent snapshot thereof |
| CN105144105A (en) * | 2013-02-27 | 2015-12-09 | 网络存储技术公司 | System and method for a scalable crash-consistent snapshot operation |
-
2017
- 2017-09-21 CN CN201710861846.8A patent/CN109542674A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1704903A (en) * | 2004-06-03 | 2005-12-07 | 华为技术有限公司 | Snapshot backup method |
| US8818951B1 (en) * | 2011-12-29 | 2014-08-26 | Emc Corporation | Distributed file system having separate data and metadata and providing a consistent snapshot thereof |
| CN102971698A (en) * | 2012-06-29 | 2013-03-13 | 华为技术有限公司 | Snapshot data-processing method and system, storage system and snapshot agency |
| CN105144105A (en) * | 2013-02-27 | 2015-12-09 | 网络存储技术公司 | System and method for a scalable crash-consistent snapshot operation |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110765071A (en) * | 2019-11-08 | 2020-02-07 | 无锡华云数据技术服务有限公司 | Method and related device for carrying out snapshot on distributed storage |
| CN110765071B (en) * | 2019-11-08 | 2022-05-06 | 华云数据控股集团有限公司 | Method and related device for carrying out snapshot on distributed storage |
| CN112965945A (en) * | 2019-12-13 | 2021-06-15 | 阿里巴巴集团控股有限公司 | Data storage method and device, electronic equipment and computer readable medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11757795B2 (en) | Resolving mediator unavailability | |
| US11099953B2 (en) | Automatic data healing using a storage controller | |
| CN106407040B (en) | A kind of duplicating remote data method and system | |
| US9740416B2 (en) | Essential metadata replication | |
| CN113485636B (en) | Data access method, device and system | |
| US10218804B2 (en) | Selective token clash checking for a data write | |
| CN106126374B (en) | Method for writing data, method for reading data and device | |
| US20150236974A1 (en) | Computer system and load balancing method | |
| US20180364948A1 (en) | Data Processing Method, Related Device, and Storage System | |
| CN103229134A (en) | Storage apparatus and control method thereof | |
| US11226746B2 (en) | Automatic data healing by I/O | |
| CN110147203A (en) | A kind of file management method, device, electronic equipment and storage medium | |
| US9514013B2 (en) | Maintaining inactive copy relationships for secondary storages of active copy relationships having a common primary storage for use in case of a failure of the common primary storage | |
| CN110502187B (en) | Snapshot rollback method and device | |
| US10469288B2 (en) | Efficient data transfer in remote mirroring connectivity on software-defined storage systems | |
| CN108304142A (en) | A kind of data managing method and device | |
| US20150381727A1 (en) | Storage functionality rule implementation | |
| CN109542674A (en) | Snapshot creation method, device, equipment and the medium of distributed system | |
| US10216534B1 (en) | Moving storage volumes for improved performance | |
| US10127270B1 (en) | Transaction processing using a key-value store | |
| US11256584B2 (en) | One-step disaster recovery configuration on software-defined storage systems | |
| US10606497B1 (en) | Systems and methods for managing replication path direction in storage networks | |
| CN111399774B (en) | Data processing method and device based on snapshot under distributed storage system | |
| WO2017067297A1 (en) | Method and device for reading and writing video data in nas device | |
| US9729629B2 (en) | Optimizing data transfer across multiple asynchronous data replication sessions using shared bandwidth |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210915 Address after: Room 508, floor 5, building 4, No. 699, Wangshang Road, Changhe street, Binjiang District, Hangzhou City, Zhejiang Province Applicant after: Alibaba (China) Co.,Ltd. Address before: Greater Cayman Islands Applicant before: ALIBABA GROUP HOLDING Ltd. |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211122 Address after: 310000 No. 12, Zhuantang science and technology economic block, Xihu District, Hangzhou City, Zhejiang Province Applicant after: Aliyun Computing Co.,Ltd. Address before: 310052 room 508, 5th floor, building 4, No. 699 Wangshang Road, Changhe street, Binjiang District, Hangzhou City, Zhejiang Province Applicant before: Alibaba (China) Co.,Ltd. |
|
| TA01 | Transfer of patent application right |